Nov 02 21:12:06.829248 ip-10-0-130-83 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 21:12:07.287147 ip-10-0-130-83 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:07.287147 ip-10-0-130-83 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 21:12:07.287147 ip-10-0-130-83 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:07.287147 ip-10-0-130-83 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294059    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294143    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294148    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294151    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294154    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.294149 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294157    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294160    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294163    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294166    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294169    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294172    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294174    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294177    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294179    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294182    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294185    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294187    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294190    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294192    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294195    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294197    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294200    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294202    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294205    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294211    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.294382 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294214    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294217    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294219    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294222    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294225    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294227    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294230    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294232    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294235    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294237    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294239    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294242    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294245    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294247    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294251    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294253    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294257    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294262    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294265    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.295073 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294267    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294270    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294272    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294276    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294278    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294281    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294284    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294286    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294289    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294292    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294294    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294297    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294299    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294302    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294305    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294308    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294310    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294313    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294316    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294319    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.295707 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294321    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294324    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294327    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294330    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294332    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294335    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294338    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294342    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294344    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294347    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294350    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294352    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294355    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294358    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294360    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294363    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294365    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294368    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294370    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.296477 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294373    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294376    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294380    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294385    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294388    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294391    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294394    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294400    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294402    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294405    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294408    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294411    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294413    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294416    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294418    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294421    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294423    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294426    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294429    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.297162 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294431    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294435    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294439    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.294441    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295284    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295292    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295297    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295302    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295305    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295307    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295310    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295313    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295315    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295318    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295321    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295324    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295326    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295329    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295331    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.297761 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295334    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295337    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295340    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295343    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295345    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295348    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295350    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295353    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295355    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295358    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295360    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295363    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295365    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295367    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295371    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295374    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295376    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295379    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295381    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.298245 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295384    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295386    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295389    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295392    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295394    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295397    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295399    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295402    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295405    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295407    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295410    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295412    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295414    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295417    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295419    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295422    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295425    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295427    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295430    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295432    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.298692 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295435    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295437    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295440    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295442    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295444    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295447    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295450    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295453    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295456    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295459    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295461    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295464    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295466    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295469    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295471    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295474    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295476    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295479    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295482    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.299215 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295485    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295487    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295490    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295493    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295495    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295498    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295500    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295503    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295506    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295508    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295511    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295514    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295517    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295519    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295522    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295525    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295527    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295529    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295532    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295535    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.299670 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295538    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295541    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295543    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295546    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295549    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295551    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295554    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295556    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295558    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295561    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295563    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295568    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.295572    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296093    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296099    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296103    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296105    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296108    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296111    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.300176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296113    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296116    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296119    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296128    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296131    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296136    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296139    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296143    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296145    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296148    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296151    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296153    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296156    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296159    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296161    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296164    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296167    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296169    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296172    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.300737 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296175    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296177    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296180    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296182    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296186    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296189    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296191    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296194    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296196    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296201    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296204    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296207    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296209    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296212    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296214    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296217    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296220    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296229    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296231    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296234    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.301214 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296237    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296239    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296242    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296244    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296247    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296250    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296253    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296255    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296258    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296261    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296263    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296266    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296268    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296271    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296274    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296276    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296278    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296281    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296283    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296286    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.301678 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296288    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296292    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296295    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296297    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296300    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296306    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296309    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296312    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296315    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296317    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296321    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296323    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296328    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296331    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296334    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296337    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296339    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296342    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296345    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.302176 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296347    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296350    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296352    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296355    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296357    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296360    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296362    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296365    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296367    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296370    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296372    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296375    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296377    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296380    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296384    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296387    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296389    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296392    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296394    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296397    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.302659 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296399    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.296401    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296505    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296520    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296527    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296532    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296537    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296541    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296545    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296550    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296555    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296558    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296562    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296566    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296569    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296572    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296575    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296578    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296581    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296584    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296589    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296592    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296595    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296599    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 21:12:07.303144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296602    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296606    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296610    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296613    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296617    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296620    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296624    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296627    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296630    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296633    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296638    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296641    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296644    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296647    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296651    2580 flags.go:64] FLAG: --enable-server="true"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296654    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296658    2580 flags.go:64] FLAG: --event-burst="100"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296662    2580 flags.go:64] FLAG: --event-qps="50"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296667    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296671    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296674    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296679    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296682    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296685    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296688    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 02 21:12:07.303750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296691    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296695    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296698    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296701    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296704    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296707    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296710    2580 flags.go:64] FLAG: --feature-gates=""
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296714    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296717    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296720    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296724    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296727    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296730    2580 flags.go:64] FLAG: --help="false"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296733    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296736    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296740    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296743    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296746    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296749    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296752    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296755    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296764    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296768    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 21:12:07.304807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296771    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296774    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296779    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296782    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296785    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296788    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296791    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296794    2580 flags.go:64] FLAG: --lock-file=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296797    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296800    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296803    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296809    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296812    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296815    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296818    2580 flags.go:64] FLAG: --logging-format="text"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296821    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296825    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296828    2580 flags.go:64] FLAG: --manifest-url=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296831    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296836    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296839    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296843    2580 flags.go:64] FLAG: --max-pods="110"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296860    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296863    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296867    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 21:12:07.305731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296870    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296872    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296876    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296879    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296887    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296890    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296899    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296903    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296906    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296910    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296913    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296916    2580 flags.go:64] FLAG: --port="10250"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296920    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296923    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-069dd70718e285386"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296926    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296929    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296932    2580 flags.go:64] FLAG: --register-node="true"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296935    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296939    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296942    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296945    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296948    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296952    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296955    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296959    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296962    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 21:12:07.306459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296965    2580 flags.go:64] FLAG: --runonce="false"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296968    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296971    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296974    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296977    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296980    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296983    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296986    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296990    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296993    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296996    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.296999    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297002    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297011    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297014    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297019    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297024    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297027    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297030    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297039    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297041    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297044    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297047    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297050    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297053    2580 flags.go:64] FLAG: --v="2"
Nov 02 21:12:07.307140 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297058    2580 flags.go:64] FLAG: --version="false"
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297063    2580 flags.go:64] FLAG: --vmodule=""
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297067    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297070    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297259    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297263    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297268    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297271    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297274    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297278    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297282    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297285    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297288    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297291    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297294    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297297    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297300    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297302    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297305    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297307    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.307731 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297310    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297319    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297323    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297326    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297329    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297331    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297334    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297337    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297339    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297342    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297345    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297347    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297350    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297353    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297356    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297358    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297361    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297363    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297367    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297369    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.308235 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297372    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297375    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297377    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297380    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297383    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297385    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297388    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297391    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297393    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297396    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297398    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297401    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297403    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297406    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297415    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297418    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297421    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297423    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297427    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.308708 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297431    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297434    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297437    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297439    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297442    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297444    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297447    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297450    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297452    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297455    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297457    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297462    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297464    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297467    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297469    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297472    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297475    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297477    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297480    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.309175 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297482    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297485    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297487    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297490    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297492    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297495    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297498    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297500    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297504    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297512    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297515    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297518    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297520    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297523    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297525    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297528    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297531    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297533    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297536    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297538    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.309639 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297541    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297543    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297546    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297548    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297552    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297555    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297558    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297561    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297564    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297566    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297569    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297571    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.297597    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297648    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297653    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297656    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.310122 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297658    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297661    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297670    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297672    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297675    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297679    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297688    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297691    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297694    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297696    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297699    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297701    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297704    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297707    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297709    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297712    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297715    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297717    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297720    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297723    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.310503 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297727    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297730    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297732    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297735    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297738    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297740    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297743    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297746    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297748    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297751    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297753    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297756    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297759    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297762    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297765    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297768    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297770    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297775    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297777    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.311110 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297786    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297789    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297791    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297796    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297800    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297803    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297806    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297809    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297812    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297814    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297817    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297819    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297822    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297826    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297828    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297831    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297833    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297836    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297839    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.311561 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297841    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297855    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297858    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297860    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297863    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297865    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297868    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297870    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297873    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297876    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297878    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297882    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297884    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297887    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297896    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297898    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297901    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297904    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297906    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297908    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.312031 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297911    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297914    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297916    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297919    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297921    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297924    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297928    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297930    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297933    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297936    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297938    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297941    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297944    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297947    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297949    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297952    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297954    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297957    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297959    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297962    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.312534 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297964    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297969    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297972    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297977    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.297979    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307013    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307033    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307049    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307055    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307121    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307127    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307130    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307133    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307136    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307139    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307143    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307145    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307148    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307152    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.313032 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307156    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307159    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307162    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307164    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307167    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307171    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307173    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307176    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307179    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307182    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307185    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307187    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307190    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307192    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307195    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307198    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307201    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307204    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307206    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307209    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.313482 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307211    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307214    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307217    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307220    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307222    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307225    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307228    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307230    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307233    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307235    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307238    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307241    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307243    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307246    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307248    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307251    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307259    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307262    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307265    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.313963 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307267    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307270    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307273    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307275    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307279    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307284    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307287    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307290    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307294    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307298    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307301    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307304    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307307    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307310    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307313    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307316    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307320    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307323    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307325    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.314412 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307328    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307330    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307333    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307336    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307338    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307341    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307344    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307346    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307349    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307351    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307354    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307363    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307366    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307369    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307372    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307374    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307377    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307379    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307382    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.314954 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307385    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307387    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307390    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307393    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307395    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307398    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307400    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307403    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307406    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307408    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307412    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307416    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307420    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307423    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307426    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307429    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307431    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307434    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.315401 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307436    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307464    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307613    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307617    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307621    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307624    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307627    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307636    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307639    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307642    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307645    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307647    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307650    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307653    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307655    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307658    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.315822 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307660    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307663    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307666    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307669    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307672    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307674    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307677    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307679    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307682    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307685    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307687    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307690    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307692    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307695    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307698    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307701    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307703    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307706    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307708    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307711    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.316229 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307713    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307716    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307718    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307721    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307729    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307732    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307735    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307737    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307740    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307743    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307747    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307752    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307755    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307758    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307761    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307763    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307766    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307770    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307773    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.316696 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307776    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307779    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307782    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307785    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307787    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307790    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307792    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307795    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307797    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307800    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307802    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307805    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307807    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307810    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307812    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307815    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307818    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307820    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307837    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307840    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.317154 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307843    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307859    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307862    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307876    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307879    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307881    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307884    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307887    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307890    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307892    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307895    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307898    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307900    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307904    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307907    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307910    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307912    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307915    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307918    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.317613 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307920    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307922    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307925    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307927    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307930    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307933    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307935    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307938    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307940    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307943    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307945    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307948    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307959    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.307962    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.307986    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308028    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:07.318082 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308032    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308035    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308040    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308044    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308050    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308053    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308056    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308059    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308062    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308064    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308067    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308071    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308074    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308077    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308081    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308084    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308087    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308089    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308092    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:07.318464 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308095    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308097    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308100    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308103    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308105    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308108    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308111    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308113    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308116    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308118    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308127    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308130    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308132    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308135    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308137    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308141    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308144    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308148    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308150    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:07.318930 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308153    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308155    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308158    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308160    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308163    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308165    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308168    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308170    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308173    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308176    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308178    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308181    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308183    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308186    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308189    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308191    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308193    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308196    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308199    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308202    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:07.319416 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308204    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308207    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308209    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308212    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308216    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308219    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308222    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308224    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308228    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308230    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308234    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308237    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308240    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308243    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308245    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308248    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308251    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308254    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308256    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:07.319911 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308259    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308261    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308264    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308267    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308269    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308272    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308274    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308277    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308280    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308282    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308285    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308287    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308290    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308292    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308295    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308298    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308300    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308303    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308307    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308309    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:07.320376 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308312    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308319    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308321    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308325    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308327    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308330    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308333    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:07.308335    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.308402    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.310956    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.311977    2580 server.go:1014] "Starting client certificate rotation"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.312075    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.312872    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.314299    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:12:07.320828 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.317827    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:12:07.331448 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.331428    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 02 21:12:07.335452 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.335437    2580 log.go:25] "Validated CRI v1 image API"
Nov 02 21:12:07.336371 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.336343    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 21:12:07.339130 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.339104    2580 fs.go:135] Filesystem UUIDs: map[598e98f0-75fe-4a3b-8fe2-e196d2e6a95f:/dev/nvme0n1p4 76d2441d-5b02-4d59-908b-ee16480da4b9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 02 21:12:07.339197 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.339128    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 21:12:07.345053 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.344917    2580 manager.go:217] Machine: {Timestamp:2026-11-02 21:12:07.343430644 +0000 UTC m=+0.382670755 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110372 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21c8cd27bb81d8b633d57d6cfc9543 SystemUUID:ec21c8cd-27bb-81d8-b633-d57d6cfc9543 BootID:5568069f-7726-4fb1-aed1-38dc23dd47a8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:ce:88:06:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:ce:88:06:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:31:84:39:8c:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 21:12:07.345673 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.345661    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 21:12:07.345773 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.345756    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 21:12:07.347857 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.347800    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 21:12:07.348070 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.347860    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-83.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 21:12:07.348173 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.348078    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 21:12:07.348173 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.348091    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 21:12:07.348173 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.348137    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:12:07.348173 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.348156    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:12:07.349510 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.349496    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 21:12:07.354260 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.354242    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 21:12:07.354404 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.354393    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 21:12:07.355267 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.355255    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 21:12:07.355331 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.355274    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 21:12:07.355331 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.355289    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 21:12:07.355331 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.355306    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 02 21:12:07.355331 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.355319    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 21:12:07.358257 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.358238    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:12:07.361176 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.361157    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 21:12:07.368716 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.368699    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:12:07.369727 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369714    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 21:12:07.369765 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369745    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 21:12:07.369933 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369922    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 21:12:07.369984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369937    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 21:12:07.369984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369943    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 21:12:07.369984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369948    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 21:12:07.369984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369955    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 21:12:07.369984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.369961    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 21:12:07.370587 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370576    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 21:12:07.370587 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370586    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 21:12:07.370663 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370593    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 21:12:07.370663 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370599    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 21:12:07.370663 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370609    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 21:12:07.370663 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.370618    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 21:12:07.372068 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.372057    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 21:12:07.372068 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.372068    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 21:12:07.376187 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.376170    2580 server.go:1263] "Started kubelet"
Nov 02 21:12:07.376246 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.376221    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 21:12:07.376366 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.376318    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 21:12:07.376428 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.376396    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 21:12:07.376952 ip-10-0-130-83 systemd[1]: Started Kubernetes Kubelet.
Nov 02 21:12:07.378464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.378448    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 21:12:07.380588 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.380568    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 21:12:07.382144 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.382128    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 21:12:07.386293 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.386272    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 21:12:07.386869 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.386840    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 21:12:07.386993 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.386975    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:07.387718 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.387699    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 21:12:07.387718 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.387717    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 21:12:07.387842 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.387699    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 21:12:07.387842 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.387788    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 21:12:07.387842 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.387794    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 21:12:07.388468 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388454    2580 factory.go:55] Registering systemd factory
Nov 02 21:12:07.388468 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388469    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 02 21:12:07.388747 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388735    2580 factory.go:153] Registering CRI-O factory
Nov 02 21:12:07.388747 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388748    2580 factory.go:223] Registration of the crio container factory successfully
Nov 02 21:12:07.388824 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388802    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 21:12:07.388897 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388835    2580 factory.go:103] Registering Raw factory
Nov 02 21:12:07.388897 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.388867    2580 manager.go:1199] Started watching for new ooms in manager
Nov 02 21:12:07.390027 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.390012    2580 manager.go:319] Starting recovery of all containers
Nov 02 21:12:07.402109 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.402084    2580 manager.go:324] Recovery completed
Nov 02 21:12:07.406598 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.406584    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.408546 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.408526    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.408614 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.408560    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.408614 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.408576    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.409226 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409211    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 02 21:12:07.409226 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409221    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.409226 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409222    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 21:12:07.409375 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409244    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 21:12:07.409947 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409929    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.410009 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409959    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.410009 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.409969    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.411437 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.411424    2580 policy_none.go:50] "Start"
Nov 02 21:12:07.412107 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.412090    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 21:12:07.412191 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.412118    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 21:12:07.414166 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.414152    2580 policy_none.go:44] "Start"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.454532    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.459632    2580 manager.go:341] "Starting Device Plugin manager"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.459672    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.459685    2580 server.go:87] "Starting device plugin registration server"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.459946    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.459955    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.460051    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.460151    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.460159    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.460735    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.460778    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.466997    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.468191    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.468208    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.468226    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 21:12:07.468503 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.468256    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 21:12:07.506620 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.506584    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-83.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 21:12:07.506620 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.506591    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-83.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 21:12:07.553023 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.551797    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-83.us-west-2.compute.internal.18e456be673a6430  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-83.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-83.us-west-2.compute.internal,},FirstTimestamp:2026-11-02 21:12:07.375954992 +0000 UTC m=+0.415195105,LastTimestamp:2026-11-02 21:12:07.375954992 +0000 UTC m=+0.415195105,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-83.us-west-2.compute.internal,}"
Nov 02 21:12:07.560888 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.560867    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.561888 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.561870    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.561962 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.561904    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.561962 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.561916    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.561962 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.561959    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.569043 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.569021    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"]
Nov 02 21:12:07.569106 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.569084    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.569914 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.569900    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.569982 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.569930    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.569982 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.569943    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.572192 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572179    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.572344 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.572386 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572363    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.572919 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572903    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.572919 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572915    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.573029 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572930    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.573029 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572943    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.573094 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.572933    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.573094 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.573065    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.573748 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.573731    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 21:12:07.573820 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.573779    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-77gdr"
Nov 02 21:12:07.575123 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.575103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.575216 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.575153    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:07.576003 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.575946    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:07.576003 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.575955    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 02 21:12:07.576003 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.575954    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-83.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 02 21:12:07.576003 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.575982    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:07.576215 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.576011    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:07.576215 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.576167    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 02 21:12:07.588756 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.588728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.588879 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.588767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.588879 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.588793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.607958 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.607928    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.612555 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.612536    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.652576 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.652547    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:07.661355 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.661335    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.661449 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.661364    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-83.us-west-2.compute.internal\": node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:07.689168 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.689276 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.689276 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.689276 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689244    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.689276 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689261    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.689411 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.689245    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.765999 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.765800    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:07.867084 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.867007    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:07.914410 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.914372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.917816 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:07.917796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:07.969584 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:07.969537    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.070514 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.070478    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.171285 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.171254    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.272453 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.272416    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.312641 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.312608    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 21:12:08.313118 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.312769    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 21:12:08.313118 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.312847    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa4a48b711f844547b37f00e18a10669-41cfdfea4362ef04.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-83.us-west-2.compute.internal.18e456be692c018c\": read tcp 10.0.130.83:42448->3.146.207.81:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-83.us-west-2.compute.internal.18e456be692c018c  default   4836 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-83.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-83.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-83.us-west-2.compute.internal,},FirstTimestamp:2026-11-02 21:12:07 +0000 UTC,LastTimestamp:2026-11-02 21:12:07.569936874 +0000 UTC m=+0.609176985,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-83.us-west-2.compute.internal,}"
Nov 02 21:12:08.373109 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.373063    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.474079 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:08.474001    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:08.558638 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.558605    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:08.579967 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.579918    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:07:08 +0000 UTC" deadline="2028-05-03 00:07:19.681630013 +0000 UTC"
Nov 02 21:12:08.579967 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.579959    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13130h55m11.101674295s"
Nov 02 21:12:08.587011 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.586973    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:08.651291 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.651262    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:12:08.653195 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.653180    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:08.718084 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.718057    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:12:08.985640 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.985610    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:08.985867 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:08.985852    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:09.201189 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.201149    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a88fcfc7e7a1dfee7d96883c60ca9fa.slice/crio-02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5 WatchSource:0}: Error finding container 02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5: Status 404 returned error can't find the container with id 02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5
Nov 02 21:12:09.201633 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.201612    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92be8601b1e73fbd81a5b1380a217ee7.slice/crio-0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8 WatchSource:0}: Error finding container 0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8: Status 404 returned error can't find the container with id 0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8
Nov 02 21:12:09.207212 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.207197    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:12:09.362238 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.362144    2580 apiserver.go:52] "Watching apiserver"
Nov 02 21:12:09.389349 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.389313    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 21:12:09.473904 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.473857    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 21:12:09.473904 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.473881    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"7a88fcfc7e7a1dfee7d96883c60ca9fa","Type":"ContainerStarted","Data":"02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5"}
Nov 02 21:12:09.474253 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.474231    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr","openshift-cluster-node-tuning-operator/tuned-94n6x","openshift-network-diagnostics/network-check-target-pg9d6","openshift-dns/node-resolver-x2blv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal","openshift-multus/multus-additional-cni-plugins-x7qwl","kube-system/global-pull-secret-syncer-pb8vt","openshift-multus/network-metrics-daemon-wlkwk","openshift-ovn-kubernetes/ovnkube-node-fsnkc","openshift-image-registry/node-ca-s6ntp","openshift-multus/multus-zmxlz","openshift-network-operator/iptables-alerter-7n46x","kube-system/konnectivity-agent-jp6br","kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"]
Nov 02 21:12:09.480159 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.478557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.480159 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.478673    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:09.481049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.481030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.483363 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.483340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:09.483464 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.483416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:09.485666 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.485651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.488021 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.487997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.488115 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.488068    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:09.490463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.490445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.493511 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.493495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.496003 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.495980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.497700 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.497781 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.497829 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.497877 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.497913 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.497961 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrsk\" (UniqueName: \"kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.497961 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498024 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.497972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:09.498061 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2ct\" (UniqueName: \"kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.498095 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.498141 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498141 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498249 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498249 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498357 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbzwm\" (UniqueName: \"kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498357 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.498357 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.498357 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.498554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.498554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498733 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498733 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.498733 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.498677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.500809 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.500794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.503465 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.503446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.505897 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.505871    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerStarted","Data":"0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8"}
Nov 02 21:12:09.505995 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.505980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.538101 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.538071    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.538101 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.538071    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 21:12:09.538291 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.538071    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.538291 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.538161    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-q4vfp\""
Nov 02 21:12:09.538291 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.538178    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 21:12:09.542230 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.542202    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.542355 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.542312    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z2vtc\""
Nov 02 21:12:09.542419 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.542389    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.577638 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.577605    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.577826 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.577652    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 21:12:09.577984 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.577967    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 21:12:09.578108 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578086    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ps8jp\""
Nov 02 21:12:09.578351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578336    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 21:12:09.578351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578345    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 21:12:09.578464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578359    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.578464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578405    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 21:12:09.578464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578408    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-crvzm\""
Nov 02 21:12:09.578464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578436    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 21:12:09.578464 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.578447    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zzckx\""
Nov 02 21:12:09.584145 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.584129    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 21:12:09.584212 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.584173    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 21:12:09.585714 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.585699    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 21:12:09.585774 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.585755    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fx97l"
Nov 02 21:12:09.588505 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.588486    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 21:12:09.599327 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.599416 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.599416 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599416 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599522 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599522 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.599522 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599520    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.599608 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599625    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrsk\" (UniqueName: \"kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599676    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.599784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7dm\" (UniqueName: \"kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.599984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600048    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh22r\" (UniqueName: \"kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600322 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600466    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.600685 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzwm\" (UniqueName: \"kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600808    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.600929    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.600935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.600989    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:10.100974066 +0000 UTC m=+3.140214189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwr9h\" (UniqueName: \"kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.601077 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfllj\" (UniqueName: \"kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2ct\" (UniqueName: \"kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601382    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601416    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.601509 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.601457    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.601534    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:10.101514954 +0000 UTC m=+3.140755069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601682    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5vq\" (UniqueName: \"kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601912    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.601916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd75v\" (UniqueName: \"kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.601991    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602180    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602215    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.602351 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.602261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.647570 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647487    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 21:12:09.647570 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647515    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.647570 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647515    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 21:12:09.647570 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647517    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.647570 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647545    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647584    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647584    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647704    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647737    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647751    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sdlzm\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647810    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647829    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dtndn\""
Nov 02 21:12:09.647838 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647843    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c5zxb\""
Nov 02 21:12:09.648114 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647897    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c6fz8\""
Nov 02 21:12:09.648114 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.647901    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 21:12:09.659105 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.659083    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:09.659156 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.659106    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:09.659156 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.659117    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:09.659234 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:09.659175    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:10.159160214 +0000 UTC m=+3.198400318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:09.659723 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.659693    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 21:12:09.663391 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.663364    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2ct\" (UniqueName: \"kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:09.663522 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.663402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrsk\" (UniqueName: \"kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.663522 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.663402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzwm\" (UniqueName: \"kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.702701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.702701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.702906 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5vq\" (UniqueName: \"kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.702906 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.702906 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702921    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702943    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702974    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.702966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703058 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd75v\" (UniqueName: \"kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703215    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703243    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703271    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.703328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703394    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703478    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703507    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.703978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7dm\" (UniqueName: \"kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703898    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh22r\" (UniqueName: \"kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.703999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704044    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704057    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704046    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704127    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.704742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704178    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704213    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704223    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704270    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704284    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704347    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704389    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704425    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.705483 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwr9h\" (UniqueName: \"kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfllj\" (UniqueName: \"kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704810    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704867    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704928    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705000    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705029    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.704273    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705137    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705135    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705191    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705376    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.706161 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705439    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705447    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705454    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705569    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705660    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705718    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705775    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.705862    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.706170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.706183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.706701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.706228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.707387 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.707366    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.707549 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.707532    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.775119 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.775081    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd75v\" (UniqueName: \"kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.777367 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.777335    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh22r\" (UniqueName: \"kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.777489 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.777471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwr9h\" (UniqueName: \"kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.777576 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.777547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7dm\" (UniqueName: \"kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.777995 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.777977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5vq\" (UniqueName: \"kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.778093 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.778046    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfllj\" (UniqueName: \"kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.792596 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.792572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:09.799352 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.799329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:09.804518 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.804496    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73de7623_757b_4d55_8782_91f9e51c88ee.slice/crio-9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187 WatchSource:0}: Error finding container 9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187: Status 404 returned error can't find the container with id 9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187
Nov 02 21:12:09.809809 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.809783    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32f0bd3_2a4b_4ff0_a531_b5c19fd5c1ee.slice/crio-c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0 WatchSource:0}: Error finding container c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0: Status 404 returned error can't find the container with id c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0
Nov 02 21:12:09.824609 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.824583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:09.831839 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.831818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:09.832989 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.832968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:09.835083 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.835054    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000770db_14e0_4f8e_aa43_2580970caff3.slice/crio-745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc WatchSource:0}: Error finding container 745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc: Status 404 returned error can't find the container with id 745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc
Nov 02 21:12:09.835669 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.835651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:09.840868 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.840845    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:09.842005 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.841984    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29dc30e2_0650_46df_8112_6a881db4dc3f.slice/crio-a689d5a4a1e227dc688b5ea8d333a4e98dcde5f7933c26f073b10a6e6414150f WatchSource:0}: Error finding container a689d5a4a1e227dc688b5ea8d333a4e98dcde5f7933c26f073b10a6e6414150f: Status 404 returned error can't find the container with id a689d5a4a1e227dc688b5ea8d333a4e98dcde5f7933c26f073b10a6e6414150f
Nov 02 21:12:09.847866 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.847832    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6836b7_7312_471d_844b_fa29f3ec0ca2.slice/crio-ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea WatchSource:0}: Error finding container ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea: Status 404 returned error can't find the container with id ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea
Nov 02 21:12:09.849090 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.849043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:09.850306 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.850287    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60379e20_430c_4731_8a53_857484f3692a.slice/crio-24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a WatchSource:0}: Error finding container 24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a: Status 404 returned error can't find the container with id 24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a
Nov 02 21:12:09.851962 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.851940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:09.854193 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:09.854175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:09.859593 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.859569    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961b46b0_0bad_47e8_ab7d_d91663021330.slice/crio-77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6 WatchSource:0}: Error finding container 77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6: Status 404 returned error can't find the container with id 77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6
Nov 02 21:12:09.864664 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.864641    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd7fca4_e943_42d4_8b24_2d765d2adc47.slice/crio-b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5 WatchSource:0}: Error finding container b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5: Status 404 returned error can't find the container with id b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5
Nov 02 21:12:09.865436 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:09.865411    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b18bd9_5d6f_478c_a3ab_164d45979936.slice/crio-d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077 WatchSource:0}: Error finding container d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077: Status 404 returned error can't find the container with id d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077
Nov 02 21:12:10.107745 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.107713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:10.107927 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.107774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:10.107927 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.107866    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:10.107927 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.107884    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:10.108069 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.107934    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:11.107921943 +0000 UTC m=+4.147162044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:10.108069 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.107949    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:11.107942592 +0000 UTC m=+4.147182694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:10.208781 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.208743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:10.208959 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.208907    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:10.208959 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.208929    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:10.208959 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.208939    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:10.209098 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:10.208991    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:11.208976302 +0000 UTC m=+4.248216407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:10.478500 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.478462    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077"}
Nov 02 21:12:10.480904 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.480872    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerStarted","Data":"ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea"}
Nov 02 21:12:10.482358 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.482331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94n6x" event={"ID":"29dc30e2-0650-46df-8112-6a881db4dc3f","Type":"ContainerStarted","Data":"a689d5a4a1e227dc688b5ea8d333a4e98dcde5f7933c26f073b10a6e6414150f"}
Nov 02 21:12:10.483801 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.483775    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zmxlz" event={"ID":"73de7623-757b-4d55-8782-91f9e51c88ee","Type":"ContainerStarted","Data":"9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187"}
Nov 02 21:12:10.485074 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.485051    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr" event={"ID":"fcd7fca4-e943-42d4-8b24-2d765d2adc47","Type":"ContainerStarted","Data":"b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5"}
Nov 02 21:12:10.486442 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.486413    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n46x" event={"ID":"961b46b0-0bad-47e8-ab7d-d91663021330","Type":"ContainerStarted","Data":"77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6"}
Nov 02 21:12:10.487807 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.487780    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s6ntp" event={"ID":"60379e20-430c-4731-8a53-857484f3692a","Type":"ContainerStarted","Data":"24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a"}
Nov 02 21:12:10.489061 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.489037    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jp6br" event={"ID":"000770db-14e0-4f8e-aa43-2580970caff3","Type":"ContainerStarted","Data":"745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc"}
Nov 02 21:12:10.490370 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.490349    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x2blv" event={"ID":"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee","Type":"ContainerStarted","Data":"c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0"}
Nov 02 21:12:10.588750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.588678    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:07:10 +0000 UTC" deadline="2028-04-09 15:56:14.947361685 +0000 UTC"
Nov 02 21:12:10.588750 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:10.588743    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12570h44m4.358623392s"
Nov 02 21:12:11.115767 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.115720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:11.115767 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.115785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:11.116048 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.115954    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:11.116048 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.116026    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:13.116005308 +0000 UTC m=+6.155245411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:11.116159 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.116096    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:11.116159 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.116133    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:13.116121393 +0000 UTC m=+6.155361499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:11.216365 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.216315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:11.216585 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.216567    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:11.216662 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.216599    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:11.216662 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.216612    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:11.216750 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.216675    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:13.216656975 +0000 UTC m=+6.255897080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:11.469440 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.469395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:11.469628 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.469586    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:11.469747 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.469734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:11.469815 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.469801    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:11.469889 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.469860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:11.470034 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:11.470013    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:11.590213 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.590140    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:07:10 +0000 UTC" deadline="2028-06-16 21:54:54.834471178 +0000 UTC"
Nov 02 21:12:11.590213 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:11.590210    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14208h42m43.244265884s"
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.139130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.139190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.139338    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.139404    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:17.139384954 +0000 UTC m=+10.178625070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.139467    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:13.139572 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.139501    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:17.139488739 +0000 UTC m=+10.178728860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:13.242144 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.242110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:13.242293 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.242277    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:13.242367 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.242302    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:13.242367 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.242315    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:13.242480 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.242372    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:17.242354109 +0000 UTC m=+10.281594226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:13.478075 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.478040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:13.478256 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.478197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:13.478336 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.478313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:13.478423 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.478399    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:13.478500 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:13.478453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:13.478605 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:13.478559    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:14.522725 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.522478    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="07b26b7123bedc61f7464cfe752dee60cb79c1da002b0231b4a06304de1e6115" exitCode=0
Nov 02 21:12:14.523701 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.523669    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"07b26b7123bedc61f7464cfe752dee60cb79c1da002b0231b4a06304de1e6115"}
Nov 02 21:12:14.535978 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.535906    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"7a88fcfc7e7a1dfee7d96883c60ca9fa","Type":"ContainerStarted","Data":"498289d121dd3da2a0af07522aeb35da6201a3fc3448a8481a70135b3cd6878b"}
Nov 02 21:12:14.545577 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.544923    2580 generic.go:356] "Generic (PLEG): container finished" podID="92be8601b1e73fbd81a5b1380a217ee7" containerID="17b086dc806a8b4566bac1127cf00d7060e42c866da2c3cc3323fb00ba9599d8" exitCode=0
Nov 02 21:12:14.545577 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.544967    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerDied","Data":"17b086dc806a8b4566bac1127cf00d7060e42c866da2c3cc3323fb00ba9599d8"}
Nov 02 21:12:14.938056 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:14.937960    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal" podStartSLOduration=5.937895086 podStartE2EDuration="5.937895086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:14.937779362 +0000 UTC m=+7.977019507" watchObservedRunningTime="2026-11-02 21:12:14.937895086 +0000 UTC m=+7.977135206"
Nov 02 21:12:15.470950 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.470847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:15.471116 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:15.471019    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:15.471175 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.471126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:15.471225 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:15.471183    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:15.471286 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.471227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:15.471336 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:15.471294    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:15.552598 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.552554    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerStarted","Data":"eb53bc9079f188187931209a861c304ef81f64e1b92c6188c4fea41c28e474a3"}
Nov 02 21:12:15.558717 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.558686    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jp6br" event={"ID":"000770db-14e0-4f8e-aa43-2580970caff3","Type":"ContainerStarted","Data":"84014520160e58e0fcd4a81f3ed6472124c5d119fc9d61f4aa7ea781484ff14d"}
Nov 02 21:12:15.797399 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:15.797278    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" podStartSLOduration=6.7972547169999995 podStartE2EDuration="6.797254717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:15.675973374 +0000 UTC m=+8.715213498" watchObservedRunningTime="2026-11-02 21:12:15.797254717 +0000 UTC m=+8.836494871"
Nov 02 21:12:16.575276 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:16.575229    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s6ntp" event={"ID":"60379e20-430c-4731-8a53-857484f3692a","Type":"ContainerStarted","Data":"51624c248267090c3550708f63ed2a0a0d86d155d8e76ca929ef14bacdfff7c7"}
Nov 02 21:12:16.697149 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:16.697085    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s6ntp" podStartSLOduration=2.406237363 podStartE2EDuration="8.697063565s" totalImagesPullingTime="6.290826202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:16.696318775 +0000 UTC m=+9.735558899" watchObservedRunningTime="2026-11-02 21:12:16.697063565 +0000 UTC m=+9.736303708"
Nov 02 21:12:16.697358 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:16.697307    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jp6br" podStartSLOduration=3.488680557 podStartE2EDuration="8.697300496s" totalImagesPullingTime="5.208619939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:15.798188272 +0000 UTC m=+8.837428398" watchObservedRunningTime="2026-11-02 21:12:16.697300496 +0000 UTC m=+9.736540621"
Nov 02 21:12:17.178386 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.178287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:17.178529 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.178483    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:17.178571 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.178548    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:25.178533106 +0000 UTC m=+18.217773206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:17.179131 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.178754    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:17.179131 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.178807    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:25.178792759 +0000 UTC m=+18.218032878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:17.179131 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.178340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:17.279755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.279322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:17.279755 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.279504    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:17.279755 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.279525    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:17.279755 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.279537    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:17.279755 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.279613    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:25.279594235 +0000 UTC m=+18.318834337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:17.472541 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.472443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:17.484554 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.483085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:17.484554 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.483176    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:17.488914 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.485161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:17.488914 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.485412    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:17.488914 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:17.485463    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:17.581419 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:17.580936    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr" event={"ID":"fcd7fca4-e943-42d4-8b24-2d765d2adc47","Type":"ContainerStarted","Data":"67e77bdb8daa095ac33da28f1598bfa58f727e1e539764da13af4d8199c0eb4d"}
Nov 02 21:12:19.427256 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.427233    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 21:12:19.470163 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.470127    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:19.470304 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:19.470276    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:19.470433 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.470346    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T21:12:19.427252305Z","UUID":"8a613a07-af56-4923-8320-5e5cdcafa781","Handler":null,"Name":"","Endpoint":""}
Nov 02 21:12:19.471197 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.471177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:19.471343 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.470889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:19.471343 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:19.471282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:19.471491 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:19.471374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:19.471834 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.471816    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 21:12:19.471921 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.471859    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 21:12:19.528445 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.528421    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:19.528797 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.528784    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:19.591875 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.590836    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zmxlz" event={"ID":"73de7623-757b-4d55-8782-91f9e51c88ee","Type":"ContainerStarted","Data":"5bf0c5f277c9d20725b88d530cf119be50fa41f9b4aa5ec3ef98586fc4216f22"}
Nov 02 21:12:19.595602 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.595572    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr" event={"ID":"fcd7fca4-e943-42d4-8b24-2d765d2adc47","Type":"ContainerStarted","Data":"fa9e68e7695facc2b25c5d062455a6dbc5509f3ab545e9328d0dd5582dd63a66"}
Nov 02 21:12:19.595732 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.595641    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:19.596127 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.596109    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:19.866877 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:19.866604    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-zmxlz" podStartSLOduration=2.530350477 podStartE2EDuration="11.866587819s" totalImagesPullingTime="9.336237342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:19.866358013 +0000 UTC m=+12.905598135" watchObservedRunningTime="2026-11-02 21:12:19.866587819 +0000 UTC m=+12.905827940"
Nov 02 21:12:20.602042 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:20.601953    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x2blv" event={"ID":"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee","Type":"ContainerStarted","Data":"9659250fc92230dfc72dc174a935cbb7d2fb131fd6d1bca33411c3411fab4d8f"}
Nov 02 21:12:20.606863 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:20.606817    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="b3460a889317646bec91c4bc8e84e84d5b920fb989a884c248c5cd5fc559f4cc" exitCode=0
Nov 02 21:12:20.607000 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:20.606915    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"b3460a889317646bec91c4bc8e84e84d5b920fb989a884c248c5cd5fc559f4cc"}
Nov 02 21:12:20.715512 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:20.715458    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-x2blv" podStartSLOduration=3.073096713 podStartE2EDuration="12.715442151s" totalImagesPullingTime="9.642345438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:20.714713678 +0000 UTC m=+13.753953799" watchObservedRunningTime="2026-11-02 21:12:20.715442151 +0000 UTC m=+13.754682272"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:21.468937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:21.468937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:21.469073    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:21.469122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:21.469223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:21.470542 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:21.469298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:21.611165 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:21.611135    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n46x" event={"ID":"961b46b0-0bad-47e8-ab7d-d91663021330","Type":"ContainerStarted","Data":"a6c992c244b5aa15d66c50e1ef3dd1f33f0406350eb6fa061d86abc548fbabd7"}
Nov 02 21:12:21.724768 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:21.724679    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7n46x" podStartSLOduration=4.309411583 podStartE2EDuration="13.724660915s" totalImagesPullingTime="9.415249332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:21.724211321 +0000 UTC m=+14.763451444" watchObservedRunningTime="2026-11-02 21:12:21.724660915 +0000 UTC m=+14.763901035"
Nov 02 21:12:22.619383 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.619295    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr" event={"ID":"fcd7fca4-e943-42d4-8b24-2d765d2adc47","Type":"ContainerStarted","Data":"e633ea287cf4fee376a654668285dace9a81b23b7923ca4affa70b1de79809ce"}
Nov 02 21:12:22.621705 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.621605    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="5738ae3f98e4d92fc609365ee0436e08f40d2db211a452e16e58ae8a09ac00c4" exitCode=0
Nov 02 21:12:22.621705 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.621647    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"5738ae3f98e4d92fc609365ee0436e08f40d2db211a452e16e58ae8a09ac00c4"}
Nov 02 21:12:22.624100 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.624054    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94n6x" event={"ID":"29dc30e2-0650-46df-8112-6a881db4dc3f","Type":"ContainerStarted","Data":"09ff85a56decd839b0f4c30a252c3e286cc3c634cf66aeee2c36ce2bed0d0e60"}
Nov 02 21:12:22.736208 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.736147    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr" podStartSLOduration=3.359841334 podStartE2EDuration="14.736132675s" totalImagesPullingTime="11.376291341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:22.73560248 +0000 UTC m=+15.774842646" watchObservedRunningTime="2026-11-02 21:12:22.736132675 +0000 UTC m=+15.775372796"
Nov 02 21:12:22.965381 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:22.965333    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-94n6x" podStartSLOduration=3.190194258 podStartE2EDuration="14.965314706s" totalImagesPullingTime="11.775120448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:22.964718321 +0000 UTC m=+16.003958444" watchObservedRunningTime="2026-11-02 21:12:22.965314706 +0000 UTC m=+16.004554843"
Nov 02 21:12:23.469299 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:23.469258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:23.469468 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:23.469267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:23.469468 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:23.469429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:23.469468 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:23.469446    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:23.469628 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:23.469571    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:23.469688 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:23.469653    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:24.631128 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:24.630833    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="2551233e037a4eb20c41cf329f76ededcacfcf739aea7b6cf6b8ec8da6d72132" exitCode=0
Nov 02 21:12:24.631128 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:24.630926    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"2551233e037a4eb20c41cf329f76ededcacfcf739aea7b6cf6b8ec8da6d72132"}
Nov 02 21:12:25.244474 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.244433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:25.244674 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.244490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:25.244674 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.244623    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:25.244779 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.244707    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret podName:baec7585-b77e-43a2-924b-3ebeb54f7406 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.244688169 +0000 UTC m=+34.283928282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret") pod "global-pull-secret-syncer-pb8vt" (UID: "baec7585-b77e-43a2-924b-3ebeb54f7406") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:12:25.244779 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.244628    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:25.244909 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.244781    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.244761342 +0000 UTC m=+34.284001445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:12:25.345581 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.345542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:25.345741 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.345724    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:12:25.345819 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.345764    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:12:25.345819 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.345778    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6sm92 for pod openshift-network-diagnostics/network-check-target-pg9d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:25.345921 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.345869    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92 podName:bcae7401-4e6b-446c-827a-74e2ba9eb8e5 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.345829843 +0000 UTC m=+34.385069959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sm92" (UniqueName: "kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92") pod "network-check-target-pg9d6" (UID: "bcae7401-4e6b-446c-827a-74e2ba9eb8e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:12:25.469325 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.469285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:25.469493 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.469295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:25.469493 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.469433    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:25.469493 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:25.469458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:25.469623 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.469565    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:25.469654 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:25.469625    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:26.639021 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.638988    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"85d5b0f48c2376ea5f9f49cd2c3434b20a27969068e9f93c81953599a654376c"}
Nov 02 21:12:26.639463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.639063    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"b8a70eb9b9f86ee3c140483a25f7b020999e3404badb84fa273329e4aa28184c"}
Nov 02 21:12:26.639463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.639099    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"9f6f8b6ad837139ac58a7a413641e510ce0fc14b60ec66c6c97285b3514f568f"}
Nov 02 21:12:26.639463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.639133    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"2c967592d8d2b0fb9840fbefaac0325de38cea4de2ab9d3920919d3400182e60"}
Nov 02 21:12:26.639463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.639162    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"1be7e44c66433c9c95a88e7786810198c2ec5bcfeeb44239e7aebb2538ae81ad"}
Nov 02 21:12:26.639463 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:26.639199    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"39a48f28028c97f17108eecec36409e6735c8b6dfa02c77843c1fdde2ce32afc"}
Nov 02 21:12:27.469324 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:27.469291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:27.469494 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:27.469400    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:27.469494 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:27.469410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:27.469494 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:27.469438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:27.469658 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:27.469513    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:27.469658 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:27.469562    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:27.644283 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:27.644248    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="28a284d2e7833bb0a918836796669ddfb14bef771c3ddd1e86165f37f0386b16" exitCode=0
Nov 02 21:12:27.644821 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:27.644289    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"28a284d2e7833bb0a918836796669ddfb14bef771c3ddd1e86165f37f0386b16"}
Nov 02 21:12:28.655018 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:28.654990    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="f35cc76ca9165e964fcbfefe4b6d385488d5b4645e214e043853b8fba76ea778" exitCode=0
Nov 02 21:12:28.655468 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:28.655052    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerDied","Data":"f35cc76ca9165e964fcbfefe4b6d385488d5b4645e214e043853b8fba76ea778"}
Nov 02 21:12:29.469287 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.469249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:29.469287 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.469274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:29.469287 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.469292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:29.469492 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:29.469381    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:29.469492 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:29.469434    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:29.469589 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:29.469512    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:29.662742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.662692    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"f81fdbcebc75ef5662678d77195fd3df219cb316aab05991499af568319ec639"}
Nov 02 21:12:29.666051 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.666028    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" event={"ID":"ed6836b7-7312-471d-844b-fa29f3ec0ca2","Type":"ContainerStarted","Data":"112760f316104cc1f4e5c70da8e1a2e5133b8e80a337d0d6b4230df74893bfe1"}
Nov 02 21:12:29.832878 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:29.832772    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x7qwl" podStartSLOduration=7.688586424 podStartE2EDuration="21.832760799s" totalImagesPullingTime="14.144174375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:29.832442795 +0000 UTC m=+22.871682916" watchObservedRunningTime="2026-11-02 21:12:29.832760799 +0000 UTC m=+22.872000920"
Nov 02 21:12:31.469182 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.468939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:31.469799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.469008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:31.469799 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:31.469295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:31.469799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.469055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:31.469799 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:31.469368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:31.469799 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:31.469429    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:31.674025 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.673988    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" event={"ID":"c3b18bd9-5d6f-478c-a3ab-164d45979936","Type":"ContainerStarted","Data":"186ac5070bb88dcb2dd97230a9b4d903cabae2696acd90eea268178ebc65afbd"}
Nov 02 21:12:31.674286 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.674263    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:31.674359 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.674328    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:31.690323 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.690300    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:31.944724 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:31.944664    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc" podStartSLOduration=7.60644877 podStartE2EDuration="23.944643511s" totalImagesPullingTime="16.338194741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:31.893869487 +0000 UTC m=+24.933109609" watchObservedRunningTime="2026-11-02 21:12:31.944643511 +0000 UTC m=+24.983883636"
Nov 02 21:12:32.676401 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:32.676369    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:32.690927 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:32.690902    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:33.404899 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.404833    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pg9d6"]
Nov 02 21:12:33.405086 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.404977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:33.405086 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:33.405063    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:33.407651 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.407631    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pb8vt"]
Nov 02 21:12:33.407742 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.407731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:33.407830 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:33.407815    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:33.408094 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.408074    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wlkwk"]
Nov 02 21:12:33.408176 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:33.408165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:33.408256 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:33.408240    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:35.468383 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:35.468359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:35.468383 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:35.468376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:35.468892 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:35.468359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:35.468892 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:35.468465    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:35.468892 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:35.468525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:35.468892 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:35.468669    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:37.469961 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:37.469927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:37.470425 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:37.470061    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb8vt" podUID="baec7585-b77e-43a2-924b-3ebeb54f7406"
Nov 02 21:12:37.470425 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:37.470081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:37.470425 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:37.470250    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pg9d6" podUID="bcae7401-4e6b-446c-827a-74e2ba9eb8e5"
Nov 02 21:12:37.470425 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:37.470290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:37.470425 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:37.470412    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wlkwk" podUID="0f074bad-307f-4832-8df7-b239864787c3"
Nov 02 21:12:39.235304 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.235143    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeReady"
Nov 02 21:12:39.235688 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.235404    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 21:12:39.347625 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.347589    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jtg5l"]
Nov 02 21:12:39.352494 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.352467    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-m9fk8"]
Nov 02 21:12:39.352620 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.352584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:39.354897 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.354872    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w69f9"]
Nov 02 21:12:39.355025 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.354960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:39.357274 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.357256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:39.392686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.392651    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:12:39.397042 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.397012    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-42685"]
Nov 02 21:12:39.397197 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.397173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.400417 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.400393    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"]
Nov 02 21:12:39.400563 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.400543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.403258 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.403237    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79fb5bb755-mzvm9"]
Nov 02 21:12:39.403397 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.403380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.406239 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.406219    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"]
Nov 02 21:12:39.406334 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.406319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.409254 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.409233    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"]
Nov 02 21:12:39.409361 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.409306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.412691 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.412672    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"]
Nov 02 21:12:39.412795 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.412769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.416311 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.416294    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"]
Nov 02 21:12:39.416590 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.416459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.419282 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.419264    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-79wcm"]
Nov 02 21:12:39.419379 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.419320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:39.422245 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.422226    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:12:39.422355 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.422333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.425578 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.425558    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-m9b57"]
Nov 02 21:12:39.425675 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.425660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.428193 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.428172    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"]
Nov 02 21:12:39.428282 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.428264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.431328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.431308    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6trh7"]
Nov 02 21:12:39.431407 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.431397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:39.435236 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.435213    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nh249"]
Nov 02 21:12:39.436822 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.436804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.439222 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.439199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.443799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.442672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkgpl\" (UniqueName: \"kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl\") pod \"kas-connection-checker-685895c7c9-m9fk8\" (UID: \"44199617-59fc-462a-9d0a-bae7f2b73599\") " pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:39.443799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.442755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5whtm\" (UniqueName: \"kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm\") pod \"kas-connection-checker-685895c7c9-jtg5l\" (UID: \"d3061070-d6da-464b-b0df-56c03513f0da\") " pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:39.443799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.442765    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.443799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.442816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qtzw\" (UniqueName: \"kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw\") pod \"kas-connection-checker-685895c7c9-w69f9\" (UID: \"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83\") " pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:39.443799 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.443073    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.444232 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.444098    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w69f9"]
Nov 02 21:12:39.446420 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.445711    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-m9fk8"]
Nov 02 21:12:39.446987 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.446715    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jtg5l"]
Nov 02 21:12:39.449459 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.449438    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-42685"]
Nov 02 21:12:39.450979 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.450961    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b8fk2\""
Nov 02 21:12:39.452732 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.452713    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"]
Nov 02 21:12:39.452834 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.452756    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"]
Nov 02 21:12:39.453588 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.453571    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"]
Nov 02 21:12:39.454153 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.454135    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 21:12:39.454924 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.454901    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:12:39.455132 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.455116    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79fb5bb755-mzvm9"]
Nov 02 21:12:39.468509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.468483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:39.468633 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.468512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:39.468633 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.468494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:39.496638 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.496610    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-m9b57"]
Nov 02 21:12:39.502402 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.502371    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"]
Nov 02 21:12:39.503247 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.503225    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"]
Nov 02 21:12:39.504583 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.504564    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6trh7"]
Nov 02 21:12:39.505610 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.505588    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:12:39.510196 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.510167    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.510290 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.510247    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.510340 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.510295    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.510479 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.510444    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 21:12:39.510886 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.510865    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.511584 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511027    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 21:12:39.511708 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511159    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.511816 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511176    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.511975 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511230    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511239    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511249    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511310    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511329    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511365    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511368    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511393    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511398    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511414    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511432    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511432    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511460    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511461    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511477    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511489    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511495    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511503    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511531    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:39.512330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.511776    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 21:12:39.543424 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.543424 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.543651 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.543706 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.543706 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.543808 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qtzw\" (UniqueName: \"kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw\") pod \"kas-connection-checker-685895c7c9-w69f9\" (UID: \"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83\") " pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:39.543808 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.543939 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.543939 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.543939 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c68dh\" (UniqueName: \"kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.543990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpc9z\" (UniqueName: \"kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4qr\" (UniqueName: \"kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr\") pod \"volume-data-source-validator-58c9f5f46d-8nshs\" (UID: \"0a750444-34a4-42f7-9262-260e8fe2cd5a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bhb\" (UniqueName: \"kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.545138 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkgpl\" (UniqueName: \"kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl\") pod \"kas-connection-checker-685895c7c9-m9fk8\" (UID: \"44199617-59fc-462a-9d0a-bae7f2b73599\") " pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbcvs\" (UniqueName: \"kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whtm\" (UniqueName: \"kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm\") pod \"kas-connection-checker-685895c7c9-jtg5l\" (UID: \"d3061070-d6da-464b-b0df-56c03513f0da\") " pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvq86\" (UniqueName: \"kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.544956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwmr\" (UniqueName: \"kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.545784 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llp4k\" (UniqueName: \"kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdjh\" (UniqueName: \"kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh\") pod \"network-check-source-6f78d59f99-l4hv8\" (UID: \"1a2829e3-5251-4728-8a16-7efd3bbe5d2c\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn2lc\" (UniqueName: \"kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk668\" (UniqueName: \"kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.546272 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.546673 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.545768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.561385 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.561358    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 21:12:39.600174 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600133    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 21:12:39.600174 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600160    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 21:12:39.600397 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600300    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 21:12:39.600512 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600496    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 21:12:39.600606 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600588    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 21:12:39.600649 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600626    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600675    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600697    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600697    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600706    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600726    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600733    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zlqgx\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600732    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 21:12:39.600755 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600757    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600766    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600759    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600760    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600791    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4znh\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600791    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 21:12:39.601045 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.600794    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 21:12:39.601410 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.601396    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nh249"]
Nov 02 21:12:39.601991 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.601977    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-79wcm"]
Nov 02 21:12:39.602544 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.602523    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"]
Nov 02 21:12:39.610260 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610244    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-q2w7t\""
Nov 02 21:12:39.610560 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610529    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hb5qv\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610593    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lplw4\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610600    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zkwqq\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610627    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vtj4n\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610645    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jzpsf\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610650    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bj87x\""
Nov 02 21:12:39.610686 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610652    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610687    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nvsgt\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610695    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mnft8\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610817    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j2kn9\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610878    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gdtlz\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610889    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dfd2\""
Nov 02 21:12:39.610973 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.610928    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jzn4w\""
Nov 02 21:12:39.614684 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.614661    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkgpl\" (UniqueName: \"kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl\") pod \"kas-connection-checker-685895c7c9-m9fk8\" (UID: \"44199617-59fc-462a-9d0a-bae7f2b73599\") " pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:39.614684 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.614683    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whtm\" (UniqueName: \"kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm\") pod \"kas-connection-checker-685895c7c9-jtg5l\" (UID: \"d3061070-d6da-464b-b0df-56c03513f0da\") " pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:39.614824 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.614707    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qtzw\" (UniqueName: \"kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw\") pod \"kas-connection-checker-685895c7c9-w69f9\" (UID: \"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83\") " pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:39.646393 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.646393 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bhb\" (UniqueName: \"kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.646520    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.646532    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.646597    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.146580208 +0000 UTC m=+33.185820306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:39.646635 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbcvs\" (UniqueName: \"kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvq86\" (UniqueName: \"kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwmr\" (UniqueName: \"kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.646983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.647049 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llp4k\" (UniqueName: \"kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdjh\" (UniqueName: \"kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh\") pod \"network-check-source-6f78d59f99-l4hv8\" (UID: \"1a2829e3-5251-4728-8a16-7efd3bbe5d2c\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn2lc\" (UniqueName: \"kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647247    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk668\" (UniqueName: \"kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.647582 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c68dh\" (UniqueName: \"kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpc9z\" (UniqueName: \"kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.647969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4qr\" (UniqueName: \"kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr\") pod \"volume-data-source-validator-58c9f5f46d-8nshs\" (UID: \"0a750444-34a4-42f7-9262-260e8fe2cd5a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:39.648330 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648078    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648122    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648188    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.148171207 +0000 UTC m=+33.187411326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648504    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648590    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.148574064 +0000 UTC m=+33.187814167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648698    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648755    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.148742126 +0000 UTC m=+33.187982225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.648919    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.648997    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:39.649102 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.649049    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.149035361 +0000 UTC m=+33.188275475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:39.649752 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.649323    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.149308448 +0000 UTC m=+33.188548567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:39.649752 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.649497    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:39.649752 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.649593    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:39.649752 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.649640    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.149625644 +0000 UTC m=+33.188865760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:39.650483 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.650457    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:39.650576 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:39.650526    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:40.150496503 +0000 UTC m=+33.189736608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:39.660820 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.660792    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.660962 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.660825    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:39.661275 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.661252    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.662072 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.661953    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.662072 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662037    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.662218 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662135    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.662218 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.662328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.662328 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662307    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.662435 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.662435 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662403    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.662435 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662416    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.662894 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662872    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.662985 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.662920    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.663105 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663050    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.663105 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663059    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.663355 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663337    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.663617 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663551    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.663617 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663585    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.663617 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663605    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.663798 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.663798 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.663733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.664095 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.664073    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.664348 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.664329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:39.664445 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.664399    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.664511 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.664446    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.665516 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.665495    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.670455 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.670434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:39.677331 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.677307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:39.704192 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.704154    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvq86\" (UniqueName: \"kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.704341 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.704319    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bhb\" (UniqueName: \"kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.704389 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.704365    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.704797 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.704779    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbcvs\" (UniqueName: \"kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:39.704885 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.704872    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.707316 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.707289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llp4k\" (UniqueName: \"kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:39.707719 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.707695    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwmr\" (UniqueName: \"kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.708181 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.708136    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:39.709012 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.708967    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpc9z\" (UniqueName: \"kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.709160 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.709136    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk668\" (UniqueName: \"kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:39.709225 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.709203    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdjh\" (UniqueName: \"kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh\") pod \"network-check-source-6f78d59f99-l4hv8\" (UID: \"1a2829e3-5251-4728-8a16-7efd3bbe5d2c\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:39.709268 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.709216    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:39.709366 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.709350    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn2lc\" (UniqueName: \"kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:39.710244 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.710228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4qr\" (UniqueName: \"kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr\") pod \"volume-data-source-validator-58c9f5f46d-8nshs\" (UID: \"0a750444-34a4-42f7-9262-260e8fe2cd5a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:39.710294 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.710239    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c68dh\" (UniqueName: \"kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:39.716124 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.716107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:39.735540 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.735517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:39.754389 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.754353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:39.763870 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.763828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:39.769117 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.769028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:39.787728 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.787685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:39.811795 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:39.811241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:40.129178 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.129151    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-m9fk8"]
Nov 02 21:12:40.130557 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.130525    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44199617_59fc_462a_9d0a_bae7f2b73599.slice/crio-a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c WatchSource:0}: Error finding container a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c: Status 404 returned error can't find the container with id a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c
Nov 02 21:12:40.131163 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.131102    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3061070_d6da_464b_b0df_56c03513f0da.slice/crio-327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75 WatchSource:0}: Error finding container 327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75: Status 404 returned error can't find the container with id 327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75
Nov 02 21:12:40.131639 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.131618    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jtg5l"]
Nov 02 21:12:40.145168 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.145080    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"]
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.156620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157288    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157353    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157334191 +0000 UTC m=+34.196574308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157425    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157436    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157470    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157458922 +0000 UTC m=+34.196699023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157525    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157556    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157546562 +0000 UTC m=+34.196786665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:40.158431 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157608    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157638    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157628663 +0000 UTC m=+34.196868764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157707    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157715    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.15773377 +0000 UTC m=+34.196973871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157796    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157828    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157818456 +0000 UTC m=+34.197058557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157915    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.15790372 +0000 UTC m=+34.197143833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157970    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:40.159226 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:40.157999    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:41.157990243 +0000 UTC m=+34.197230359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:40.176243 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.176216    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w69f9"]
Nov 02 21:12:40.176243 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.176233    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c1d3c5_0f5b_4ae5_ab02_6aa9db4d1f83.slice/crio-c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0 WatchSource:0}: Error finding container c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0: Status 404 returned error can't find the container with id c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0
Nov 02 21:12:40.184779 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.184412    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-42685"]
Nov 02 21:12:40.185292 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.185245    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921fd5a0_8fdb_4da9_a7bd_8debf8c5bdcd.slice/crio-747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6 WatchSource:0}: Error finding container 747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6: Status 404 returned error can't find the container with id 747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6
Nov 02 21:12:40.226003 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.225622    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"]
Nov 02 21:12:40.231221 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.231128    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbea11f1_0e1c_4cdd_be0e_c33538c05fee.slice/crio-013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7 WatchSource:0}: Error finding container 013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7: Status 404 returned error can't find the container with id 013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7
Nov 02 21:12:40.249262 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.249218    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"]
Nov 02 21:12:40.255222 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.255183    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2829e3_5251_4728_8a16_7efd3bbe5d2c.slice/crio-0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4 WatchSource:0}: Error finding container 0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4: Status 404 returned error can't find the container with id 0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4
Nov 02 21:12:40.284509 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.284485    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"]
Nov 02 21:12:40.287643 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.287505    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nh249"]
Nov 02 21:12:40.288643 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.288613    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-79wcm"]
Nov 02 21:12:40.289225 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:40.289200    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc1883d_1d29_4185_9ea6_fda6b9e021d1.slice/crio-126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06 WatchSource:0}: Error finding container 126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06: Status 404 returned error can't find the container with id 126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06
Nov 02 21:12:40.695389 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.695351    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-w69f9" event={"ID":"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83","Type":"ContainerStarted","Data":"a786bf74b10ca31bb2920f0f1a279984969e0098eb98ecbaec0e109690f10b43"}
Nov 02 21:12:40.695583 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.695402    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-w69f9" event={"ID":"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83","Type":"ContainerStarted","Data":"c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0"}
Nov 02 21:12:40.696567 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.696540    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9" event={"ID":"7b91894b-7a38-4d8f-be77-d76177da71bb","Type":"ContainerStarted","Data":"d4c3fd60e6b6e1e4a9fba3bcdb4ecff572a47cbbff200839a39002d8f92b9d38"}
Nov 02 21:12:40.697919 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.697899    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-jtg5l" event={"ID":"d3061070-d6da-464b-b0df-56c03513f0da","Type":"ContainerStarted","Data":"a138961360b6b097838b40ddc09afddb7b0b90eb125bba6d1702412dae673580"}
Nov 02 21:12:40.698012 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.697933    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-jtg5l" event={"ID":"d3061070-d6da-464b-b0df-56c03513f0da","Type":"ContainerStarted","Data":"327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75"}
Nov 02 21:12:40.698954 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.698932    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs" event={"ID":"0a750444-34a4-42f7-9262-260e8fe2cd5a","Type":"ContainerStarted","Data":"58ac098165c8d0d81139430c9110f7fa9f8d380a93ad35cd1527f826a58cc889"}
Nov 02 21:12:40.699994 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.699970    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8" event={"ID":"1a2829e3-5251-4728-8a16-7efd3bbe5d2c","Type":"ContainerStarted","Data":"0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4"}
Nov 02 21:12:40.701020 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.700997    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-42685" event={"ID":"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd","Type":"ContainerStarted","Data":"747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6"}
Nov 02 21:12:40.702091 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.702073    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerStarted","Data":"59c2689c6131bf09663f0d1b01142ba0dd19e907b8e747f47d977efe14ebc686"}
Nov 02 21:12:40.703198 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.703175    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nh249" event={"ID":"3cc1883d-1d29-4185-9ea6-fda6b9e021d1","Type":"ContainerStarted","Data":"126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06"}
Nov 02 21:12:40.704263 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.704244    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw" event={"ID":"cbea11f1-0e1c-4cdd-be0e-c33538c05fee","Type":"ContainerStarted","Data":"013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7"}
Nov 02 21:12:40.705503 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.705482    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-m9fk8" event={"ID":"44199617-59fc-462a-9d0a-bae7f2b73599","Type":"ContainerStarted","Data":"1f2f27bdede36351fca4c367ebf6490b123247c106c20b215cbf17808b771e5c"}
Nov 02 21:12:40.705572 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:40.705518    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-m9fk8" event={"ID":"44199617-59fc-462a-9d0a-bae7f2b73599","Type":"ContainerStarted","Data":"a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c"}
Nov 02 21:12:41.166373 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166494    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166550    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.166543532 +0000 UTC m=+36.205783652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:41.166618 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166579    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166585    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.16657487 +0000 UTC m=+36.205814971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166592    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166668    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166703    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.166681394 +0000 UTC m=+36.205921499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166728    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.16672036 +0000 UTC m=+36.205960459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:41.166895 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.166877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166910    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.166896571 +0000 UTC m=+36.206136691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166925    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166974    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.166962673 +0000 UTC m=+36.206202776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166974    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166985    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.167011    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.167003678 +0000 UTC m=+36.206243778 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.166987    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:41.167261 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.167046    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:43.167037098 +0000 UTC m=+36.206277232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:41.268219 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.268184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:41.268667 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.268233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:41.268667 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.268364    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:41.268667 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:41.268435    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:13.268414786 +0000 UTC m=+66.307654905 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:41.270791 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.270768    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:41.326653 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.326536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:41.369399 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.369366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:41.375916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.375833    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:41.533162 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.532294    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-m9fk8" podStartSLOduration=446.532274675 podStartE2EDuration="7m26.532274675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:41.530969316 +0000 UTC m=+34.570209456" watchObservedRunningTime="2026-11-02 21:12:41.532274675 +0000 UTC m=+34.571514797"
Nov 02 21:12:41.617110 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.617076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:41.643041 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.642980    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-jtg5l" podStartSLOduration=446.642961166 podStartE2EDuration="7m26.642961166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:41.642652388 +0000 UTC m=+34.681892507" watchObservedRunningTime="2026-11-02 21:12:41.642961166 +0000 UTC m=+34.682201289"
Nov 02 21:12:41.713557 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.713530    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pb8vt"]
Nov 02 21:12:41.776983 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:41.766479    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-w69f9" podStartSLOduration=446.766459797 podStartE2EDuration="7m26.766459797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:41.763223771 +0000 UTC m=+34.802463911" watchObservedRunningTime="2026-11-02 21:12:41.766459797 +0000 UTC m=+34.805699917"
Nov 02 21:12:42.073734 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:42.072818    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pg9d6"]
Nov 02 21:12:42.082244 ip-10-0-130-83 kubenswrapper[2580]: W1102 21:12:42.082204    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcae7401_4e6b_446c_827a_74e2ba9eb8e5.slice/crio-c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38 WatchSource:0}: Error finding container c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38: Status 404 returned error can't find the container with id c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38
Nov 02 21:12:42.716691 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:42.716650    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pg9d6" event={"ID":"bcae7401-4e6b-446c-827a-74e2ba9eb8e5","Type":"ContainerStarted","Data":"c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38"}
Nov 02 21:12:42.718585 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:42.718548    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pb8vt" event={"ID":"baec7585-b77e-43a2-924b-3ebeb54f7406","Type":"ContainerStarted","Data":"d89ca3fe8c57fb8ddc57c483e19f886cac048145dcf4cc81c8cf3cb995c15e29"}
Nov 02 21:12:43.187802 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.187738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:43.187997 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.187820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:43.187997 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.187896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:43.187997 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.187933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:43.187997 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.187982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:43.188210 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.188024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:43.188210 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.188051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:43.188210 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.188095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:43.188361 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188236    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:43.188361 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188298    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.188280507 +0000 UTC m=+40.227520623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:43.188732 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188707    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:43.188829 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.1887536 +0000 UTC m=+40.227993713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:43.188916 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188840    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:43.188916 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188863    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:43.188916 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188905    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.188886551 +0000 UTC m=+40.228126665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:43.189046 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188957    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:43.189046 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.188992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.188982879 +0000 UTC m=+40.228222993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:43.189046 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189038    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:43.189132 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189066    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.189056342 +0000 UTC m=+40.228296455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:43.189132 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189125    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:43.189225 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189134    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:43.189225 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.189153454 +0000 UTC m=+40.228393567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:43.189225 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189213    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:43.189316 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189243    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.18923298 +0000 UTC m=+40.228473080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:43.189316 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:43.189312    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:47.189302377 +0000 UTC m=+40.228542493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:43.734769 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.734730    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs" event={"ID":"0a750444-34a4-42f7-9262-260e8fe2cd5a","Type":"ContainerStarted","Data":"03ffe4ecf9c82373f9ecf41e42ea955bb377527a74a30b91d7e5cd3a6a3d6c61"}
Nov 02 21:12:43.849401 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:43.849331    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs" podStartSLOduration=418.933568832 podStartE2EDuration="7m1.849310544s" totalImagesPullingTime="2.915741712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:43.848333149 +0000 UTC m=+36.887573297" watchObservedRunningTime="2026-11-02 21:12:43.849310544 +0000 UTC m=+36.888550665"
Nov 02 21:12:45.751652 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:45.750809    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9" event={"ID":"7b91894b-7a38-4d8f-be77-d76177da71bb","Type":"ContainerStarted","Data":"5590c0f49257a88c03b580f419359ad9f8eedf152db668e21175c03f95d5da5e"}
Nov 02 21:12:45.866382 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:45.865883    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9" podStartSLOduration=418.526644722 podStartE2EDuration="7m3.865860287s" totalImagesPullingTime="5.339215565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:45.864401493 +0000 UTC m=+38.903641614" watchObservedRunningTime="2026-11-02 21:12:45.865860287 +0000 UTC m=+38.905100405"
Nov 02 21:12:46.767972 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.767894    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-42685" event={"ID":"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd","Type":"ContainerStarted","Data":"e7a29e01412a7b176f301639f1de8afe8d33fc87b27eeece1ea12b6707012885"}
Nov 02 21:12:46.772715 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.772691    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/0.log"
Nov 02 21:12:46.772885 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.772747    2580 generic.go:356] "Generic (PLEG): container finished" podID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerID="76230cab85754fdc5328f5c778e36227f927d11f381aded0340cd8bed86c758b" exitCode=255
Nov 02 21:12:46.772885 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.772819    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerDied","Data":"76230cab85754fdc5328f5c778e36227f927d11f381aded0340cd8bed86c758b"}
Nov 02 21:12:46.773286 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.773236    2580 scope.go:122] "RemoveContainer" containerID="76230cab85754fdc5328f5c778e36227f927d11f381aded0340cd8bed86c758b"
Nov 02 21:12:46.779671 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.779636    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw" event={"ID":"cbea11f1-0e1c-4cdd-be0e-c33538c05fee","Type":"ContainerStarted","Data":"00d9b3fc4b038bd537d3471559eb0c780be37fa3c6c24c162eea5de9682c94ca"}
Nov 02 21:12:46.889820 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.888697    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-42685" podStartSLOduration=418.977586026 podStartE2EDuration="7m4.888676517s" totalImagesPullingTime="5.911090491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:46.882314336 +0000 UTC m=+39.921554484" watchObservedRunningTime="2026-11-02 21:12:46.888676517 +0000 UTC m=+39.927916638"
Nov 02 21:12:46.999706 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:46.999117    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw" podStartSLOduration=418.765373128 podStartE2EDuration="7m4.999098592s" totalImagesPullingTime="6.233725464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:46.995157008 +0000 UTC m=+40.034397152" watchObservedRunningTime="2026-11-02 21:12:46.999098592 +0000 UTC m=+40.038338716"
Nov 02 21:12:47.237062 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:47.237235 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:47.237235 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:47.237235 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:47.237235 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:47.237235 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:47.237476 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:47.237476 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.237312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:47.237567 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.237476    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:47.237567 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.237492    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:47.237567 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.237552    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.237534586 +0000 UTC m=+48.276774708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.237985    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238001    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238026    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238040    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238024025 +0000 UTC m=+48.277264144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.237984    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238071    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.23804968 +0000 UTC m=+48.277289789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238096    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238083276 +0000 UTC m=+48.277323374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238112    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238102568 +0000 UTC m=+48.277342668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238152    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:47.238193 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238171    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:47.238958 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238217    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238202415 +0000 UTC m=+48.277442518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:47.238958 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238243    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238227991 +0000 UTC m=+48.277468110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:47.238958 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238279    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:47.238958 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.238348    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.238334035 +0000 UTC m=+48.277574137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:47.547485 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.547411    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sbzp5"]
Nov 02 21:12:47.575339 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.575306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.608031 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.607947    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sbzp5"]
Nov 02 21:12:47.627727 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.627427    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 21:12:47.627727 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.627530    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mhk5v\""
Nov 02 21:12:47.646191 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.646155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.646377 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.646292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7q4\" (UniqueName: \"kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.646377 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.646349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.646465 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.646390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.646568 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.646541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.716916 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.716882    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 21:12:47.747770 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.747738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.747927 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.747822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.747927 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.747903    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.748023 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.748012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.748106 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.748083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7q4\" (UniqueName: \"kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.748156 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.748137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.748270 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.748246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:47.748310 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.748272    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:47.748350 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.748330    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 21:12:47.748384 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.748334    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:48.248315768 +0000 UTC m=+41.287555881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:47.748436 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.748387    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:48.248372094 +0000 UTC m=+41.287612198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : configmap "kube-rbac-proxy" not found
Nov 02 21:12:47.792553 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.792524    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:47.793079 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.793058    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/0.log"
Nov 02 21:12:47.793143 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.793100    2580 generic.go:356] "Generic (PLEG): container finished" podID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a" exitCode=255
Nov 02 21:12:47.793273 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.793238    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerDied","Data":"e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"}
Nov 02 21:12:47.793405 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.793330    2580 scope.go:122] "RemoveContainer" containerID="76230cab85754fdc5328f5c778e36227f927d11f381aded0340cd8bed86c758b"
Nov 02 21:12:47.793885 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.793842    2580 scope.go:122] "RemoveContainer" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"
Nov 02 21:12:47.794076 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:47.794049    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:12:47.813574 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:47.813461    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7q4\" (UniqueName: \"kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:48.253365 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.253331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:48.253473 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.253426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:48.253654 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:48.253637    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:48.253722 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:48.253710    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:49.253693074 +0000 UTC m=+42.292933175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:48.253765 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:48.253757    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 21:12:48.253794 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:48.253790    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:49.253779019 +0000 UTC m=+42.293019132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : configmap "kube-rbac-proxy" not found
Nov 02 21:12:48.798280 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.798251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:48.798768 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.798675    2580 scope.go:122] "RemoveContainer" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"
Nov 02 21:12:48.798978 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:48.798918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:12:48.800313 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.800273    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nh249" event={"ID":"3cc1883d-1d29-4185-9ea6-fda6b9e021d1","Type":"ContainerStarted","Data":"c44da5b55581e4f53132f26f790fc11292c60c9eeca93b5af4963fef50fefd94"}
Nov 02 21:12:48.801905 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.801877    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pg9d6" event={"ID":"bcae7401-4e6b-446c-827a-74e2ba9eb8e5","Type":"ContainerStarted","Data":"b46923e4fe16f12625783d1885956f22138d8557546b9e43c377dbcd4008906c"}
Nov 02 21:12:48.802010 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.801942    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:48.803400 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:48.803379    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8" event={"ID":"1a2829e3-5251-4728-8a16-7efd3bbe5d2c","Type":"ContainerStarted","Data":"561d0a1e6bc701c64774f2a8c3a9a0ce0e70d9b2b13f6abe1527d3c029d9e9e5"}
Nov 02 21:12:49.138121 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.138025    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8" podStartSLOduration=415.650068425 podStartE2EDuration="7m3.138013059s" totalImagesPullingTime="7.487944634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:49.137631306 +0000 UTC m=+42.176871426" watchObservedRunningTime="2026-11-02 21:12:49.138013059 +0000 UTC m=+42.177253180"
Nov 02 21:12:49.138897 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.138841    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pg9d6" podStartSLOduration=34.555275102 podStartE2EDuration="41.138830067s" totalImagesPullingTime="6.583554965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:49.025948727 +0000 UTC m=+42.065188850" watchObservedRunningTime="2026-11-02 21:12:49.138830067 +0000 UTC m=+42.178070188"
Nov 02 21:12:49.254870 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.254790    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nh249" podStartSLOduration=2.292921993 podStartE2EDuration="10.254776495s" totalImagesPullingTime="7.961854502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:12:49.252703543 +0000 UTC m=+42.291943690" watchObservedRunningTime="2026-11-02 21:12:49.254776495 +0000 UTC m=+42.294016620"
Nov 02 21:12:49.263386 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.263345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:49.263547 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.263529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:49.263696 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:49.263665    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 21:12:49.263831 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:49.263741    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:51.263722411 +0000 UTC m=+44.302962512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : configmap "kube-rbac-proxy" not found
Nov 02 21:12:49.264195 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:49.264179    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:49.264272 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:49.264230    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:51.264216757 +0000 UTC m=+44.303456870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:49.769440 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.769401    2580 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:49.769592 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.769460    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:49.806371 ip-10-0-130-83 kubenswrapper[2580]: I1102 21:12:49.806338    2580 scope.go:122] "RemoveContainer" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"
Nov 02 21:12:49.806748 ip-10-0-130-83 kubenswrapper[2580]: E1102 21:12:49.806544    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:12:50.398338 ip-10-0-130-83 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 21:12:50.412329 ip-10-0-130-83 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 21:12:50.412559 ip-10-0-130-83 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 21:12:50.412621 ip-10-0-130-83 systemd[1]: kubelet.service: Consumed 3.071s CPU time.
Nov 02 21:12:50.424588 ip-10-0-130-83 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 21:12:50.512955 ip-10-0-130-83 kubenswrapper[4888]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:50.512955 ip-10-0-130-83 kubenswrapper[4888]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 21:12:50.512955 ip-10-0-130-83 kubenswrapper[4888]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:50.512955 ip-10-0-130-83 kubenswrapper[4888]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:12:50.515143 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515054    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.515143 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515143    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.515143 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515147    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515150    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515153    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515156    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515159    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515162    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515165    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515168    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515171    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515173    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515176    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515178    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515181    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515184    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515186    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515189    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515191    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515194    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515196    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.515242 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515199    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515202    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515211    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515214    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515217    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515220    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515222    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515225    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515228    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515230    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515234    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515239    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515242    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515246    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515249    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515252    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515254    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515257    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515260    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.515799 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515263    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515265    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515268    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515271    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515273    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515276    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515278    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515281    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515287    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515290    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515292    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515295    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515298    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515301    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515304    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515306    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515315    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515317    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515320    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515323    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.516553 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515325    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515328    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515330    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515333    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515336    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515339    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515342    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515344    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515347    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515349    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515352    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515354    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515357    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515359    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515362    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515364    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515367    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515370    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515372    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.517244 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515374    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515380    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515384    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515387    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515390    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515393    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515395    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515398    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515401    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515403    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515406    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515409    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515411    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515414    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515416    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515419    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515422    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515425    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515428    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.517698 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515430    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515433    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515436    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515438    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515441    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515443    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515447    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.515450    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517400    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517416    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517419    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517423    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517427    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517431    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517434    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517437    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517439    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517444    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517449    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517452    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.518181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517455    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517458    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517460    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517463    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517466    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517468    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517471    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517474    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517478    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517481    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517483    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517486    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517488    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517491    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517494    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517497    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517500    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517503    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517506    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517508    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.518658 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517511    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517514    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517516    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517519    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517522    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517524    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517527    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517530    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517532    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517535    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517539    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517541    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517544    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517546    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517549    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517552    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517554    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517557    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517560    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.519193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517562    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517565    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517568    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517570    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517573    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517576    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517582    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517585    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517587    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517590    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517592    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517595    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517597    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517600    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517602    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517605    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517608    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517611    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517614    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517617    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.519659 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517620    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517623    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517625    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517628    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517631    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517634    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517636    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517639    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517641    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517644    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517647    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517649    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517660    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517663    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517667    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517670    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517674    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517677    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517679    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517682    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.520177 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517684    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517689    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517693    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517696    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517698    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517701    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517704    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517707    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517715    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517719    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517722    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517725    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517728    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517731    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.517733    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518271    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518278    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518281    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518284    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518287    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.520701 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518290    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518293    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518296    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518299    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518302    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518305    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518308    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518311    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518314    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518316    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518319    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518321    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518324    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518327    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518329    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518332    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518334    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518337    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518340    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.521232 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518342    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518345    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518347    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518351    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518354    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518356    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518359    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518362    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518364    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518367    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518370    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518372    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518375    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518377    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518380    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518382    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518385    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518388    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518391    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518394    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.521730 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518396    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518399    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518402    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518404    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518407    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518410    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518412    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518415    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518417    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518419    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518422    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518425    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518428    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518431    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518433    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518440    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518443    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518446    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518450    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.522267 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518453    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518456    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518459    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518462    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518464    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518467    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518470    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518472    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518475    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518478    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518480    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518483    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518486    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518489    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518492    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518495    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518497    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518500    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518502    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518505    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.522746 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518507    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518510    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518512    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518515    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518518    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518520    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518523    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518525    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518530    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518533    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518535    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518538    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518540    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518545    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518549    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518552    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518555    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518559    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518562    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.523301 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518564    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518567    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518570    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.518572    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518661    4888 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518669    4888 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518692    4888 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518698    4888 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518702    4888 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518705    4888 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518710    4888 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518714    4888 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518718    4888 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518721    4888 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518724    4888 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518728    4888 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518731    4888 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518734    4888 flags.go:64] FLAG: --cgroup-root=""
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518737    4888 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518740    4888 flags.go:64] FLAG: --client-ca-file=""
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518743    4888 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518748    4888 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518753    4888 flags.go:64] FLAG: --cluster-domain=""
Nov 02 21:12:50.523753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518756    4888 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518759    4888 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518762    4888 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518766    4888 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518770    4888 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518773    4888 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518776    4888 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518782    4888 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518785    4888 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518788    4888 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518791    4888 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518800    4888 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518803    4888 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518811    4888 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518815    4888 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518818    4888 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518821    4888 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518824    4888 flags.go:64] FLAG: --enable-server="true"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518827    4888 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518832    4888 flags.go:64] FLAG: --event-burst="100"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518835    4888 flags.go:64] FLAG: --event-qps="50"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518838    4888 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518841    4888 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518858    4888 flags.go:64] FLAG: --eviction-hard=""
Nov 02 21:12:50.524315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518866    4888 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518870    4888 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518874    4888 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518878    4888 flags.go:64] FLAG: --eviction-soft=""
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518883    4888 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518887    4888 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518892    4888 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518895    4888 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518898    4888 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518901    4888 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518904    4888 flags.go:64] FLAG: --feature-gates=""
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518909    4888 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518913    4888 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518916    4888 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518919    4888 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518924    4888 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518927    4888 flags.go:64] FLAG: --help="false"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518930    4888 flags.go:64] FLAG: --hostname-override="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518933    4888 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518937    4888 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518940    4888 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518943    4888 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518947    4888 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 21:12:50.524929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518950    4888 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518953    4888 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518957    4888 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518960    4888 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518963    4888 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518966    4888 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518969    4888 flags.go:64] FLAG: --kube-reserved=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518972    4888 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518975    4888 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518978    4888 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518981    4888 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518984    4888 flags.go:64] FLAG: --lock-file=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518987    4888 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518990    4888 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518993    4888 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.518999    4888 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519003    4888 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519006    4888 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519009    4888 flags.go:64] FLAG: --logging-format="text"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519012    4888 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519015    4888 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519018    4888 flags.go:64] FLAG: --manifest-url=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519021    4888 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519026    4888 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519031    4888 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 21:12:50.525494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519035    4888 flags.go:64] FLAG: --max-pods="110"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519038    4888 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519041    4888 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519044    4888 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519047    4888 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519050    4888 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519054    4888 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519057    4888 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519064    4888 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519067    4888 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519071    4888 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519074    4888 flags.go:64] FLAG: --pod-cidr=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519077    4888 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519080    4888 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519083    4888 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519086    4888 flags.go:64] FLAG: --port="10250"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519089    4888 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519092    4888 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-069dd70718e285386"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519095    4888 flags.go:64] FLAG: --qos-reserved=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519098    4888 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519101    4888 flags.go:64] FLAG: --register-node="true"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519104    4888 flags.go:64] FLAG: --register-with-taints=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519111    4888 flags.go:64] FLAG: --registry-burst="10"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519114    4888 flags.go:64] FLAG: --registry-qps="5"
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519117    4888 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519120    4888 flags.go:64] FLAG: --reserved-memory=""
Nov 02 21:12:50.526119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519125    4888 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519128    4888 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519132    4888 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519135    4888 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519137    4888 flags.go:64] FLAG: --runonce="false"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519142    4888 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519145    4888 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519148    4888 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519151    4888 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519154    4888 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519157    4888 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519161    4888 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519164    4888 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519167    4888 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519171    4888 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519174    4888 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519177    4888 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519180    4888 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519183    4888 flags.go:64] FLAG: --system-cgroups=""
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519187    4888 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519192    4888 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519195    4888 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519198    4888 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519204    4888 flags.go:64] FLAG: --tls-min-version=""
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519207    4888 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 21:12:50.526802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519210    4888 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519213    4888 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519216    4888 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519219    4888 flags.go:64] FLAG: --v="2"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519224    4888 flags.go:64] FLAG: --version="false"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519228    4888 flags.go:64] FLAG: --vmodule=""
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519232    4888 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519236    4888 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519375    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519379    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519382    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519384    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519388    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519391    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519394    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519396    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519399    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519402    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519404    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519407    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519409    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.527494 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519412    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519415    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519417    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519420    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519424    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519427    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519429    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519432    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519435    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519438    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519441    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519443    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519446    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519449    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519451    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519454    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519456    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519459    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519461    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519464    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.528029 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519466    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519469    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519472    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519476    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519480    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519484    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519487    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519490    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519493    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519496    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519498    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519501    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519503    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519506    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519509    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519511    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519514    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519517    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519520    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.528737 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519522    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519525    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519527    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519530    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519532    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519535    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519537    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519540    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519542    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519545    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519548    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519550    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519553    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519556    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519559    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519561    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519565    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519567    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519570    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.529430 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519573    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519575    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519577    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519580    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519583    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519585    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519588    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519590    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519593    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519595    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519598    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519600    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519604    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519607    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519611    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519615    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519618    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519622    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519624    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.529938 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519627    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519630    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519632    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519635    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519638    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519640    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519643    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519646    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519648    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519651    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519655    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519658    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519661    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519664    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519667    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519670    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.530416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.519697    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519742    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519746    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519749    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519752    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519755    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519758    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519761    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519764    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519766    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519770    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519772    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519776    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519779    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519781    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519784    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519788    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519792    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519796    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.530824 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519799    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519802    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519805    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519808    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519811    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519813    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519816    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519819    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519822    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519825    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519827    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519830    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519833    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519835    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519838    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519840    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519843    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519862    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519865    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519868    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.531981 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519870    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519873    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519876    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519878    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519882    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519888    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519891    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519894    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519897    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519900    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519902    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519905    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519908    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519910    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519913    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519915    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519918    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519920    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519923    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519926    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.532557 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519929    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519931    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519934    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519937    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519939    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519942    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519946    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519950    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519953    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519955    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519958    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519960    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519963    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519966    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519968    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519971    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519974    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519978    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519981    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.533146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519984    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519987    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519990    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519993    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519995    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.519998    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520001    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520004    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520007    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520010    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520012    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520015    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520017    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520020    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520023    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520025    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520028    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520031    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520033    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.533655 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520036    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520038    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520041    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520044    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520046    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520048    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520051    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520055    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520058    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.520060    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.523887    4888 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.523905    4888 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.523924    4888 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.523930    4888 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524010    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524015    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524019    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524022    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524026    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524029    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.534131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524032    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524035    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524038    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524040    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524043    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524046    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524049    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524051    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524054    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524057    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524060    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524063    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524065    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524068    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524071    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524074    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524076    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524079    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524082    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524085    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.534615 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524087    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524090    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524093    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524097    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524100    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524103    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524105    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524108    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524111    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524113    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524116    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524118    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524121    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524124    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524126    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524129    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524131    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524134    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524136    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.535179 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524139    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524142    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524145    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524148    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524150    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524153    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524156    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524158    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524161    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524163    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524166    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524168    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524171    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524173    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524176    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524179    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524182    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524185    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524188    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524190    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.535642 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524193    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524195    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524198    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524200    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524203    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524206    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524209    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524211    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524214    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524216    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524219    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524222    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524224    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524228    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524232    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524236    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524239    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524242    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524245    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.536453 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524248    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524252    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524255    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524258    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524261    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524264    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524266    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524268    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524271    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524274    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524276    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524279    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524282    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524285    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524287    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524290    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524292    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524295    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524297    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524300    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.537299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524303    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524305    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.524333    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524490    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524495    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524499    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524502    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524505    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524508    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524511    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524513    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524516    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524518    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524521    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524524    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524526    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.538157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524529    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524531    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524534    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524537    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524539    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524543    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524548    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524551    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524554    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524556    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524559    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524561    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524564    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524566    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524569    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524571    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524574    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524576    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524579    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.538875 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524581    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524584    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524587    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524589    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524592    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524595    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524598    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524600    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524603    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524605    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524608    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524610    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524613    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524615    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524617    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524620    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524623    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524625    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524628    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524631    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.539670 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524633    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524636    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524639    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524641    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524644    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524646    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524649    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524651    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524654    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524656    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524659    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524661    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524664    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524669    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524671    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524674    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524677    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524680    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524682    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.540565 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524685    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524687    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524690    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524692    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524695    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524698    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524700    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524703    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524709    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524712    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524715    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524718    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524721    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524724    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524726    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524729    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524732    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524735    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524737    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.541520 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524740    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524742    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524745    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524747    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524750    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524753    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524755    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524759    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524762    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524764    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524767    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524770    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524773    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524775    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524778    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524781    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.542363 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.524804    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524835    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524839    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524842    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524864    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524869    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524871    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524874    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524877    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524879    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524882    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524885    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524887    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524890    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524893    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524896    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524899    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524901    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524904    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524907    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:12:50.543136 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524910    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524912    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524915    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524918    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524922    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524925    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524927    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524930    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524935    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524939    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524943    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524947    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524951    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524955    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524959    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524963    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524967    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524977    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524981    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:12:50.544172 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524987    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524993    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.524997    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525002    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525006    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525010    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525014    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525017    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525022    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525026    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525030    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525034    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525038    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525042    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525046    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525050    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525054    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525061    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525066    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525070    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:12:50.545198 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525075    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525078    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525083    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525087    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525091    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525096    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525100    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525104    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525109    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525112    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525118    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525120    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525123    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525126    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525129    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525132    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525134    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525137    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525140    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:12:50.546049 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525142    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525145    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525148    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525151    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525154    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525157    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525159    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525162    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525165    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525168    4888 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525172    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525175    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525178    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525180    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525183    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525185    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525188    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525191    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525194    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:12:50.546778 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525197    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525199    4888 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525202    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525206    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525211    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525214    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525217    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525220    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525223    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:12:50.525226    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.525291    4888 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526342    4888 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526408    4888 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526594    4888 server.go:1014] "Starting client certificate rotation"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526738    4888 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526796    4888 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:07:08 +0000 UTC" deadline="2028-04-09 02:53:27.235089919 +0000 UTC"
Nov 02 21:12:50.547299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.526862    4888 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12557h40m36.708248741s"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.527115    4888 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.528183    4888 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.529439    4888 log.go:25] "Validated CRI v1 runtime API"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.532379    4888 log.go:25] "Validated CRI v1 image API"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.533091    4888 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 21:12:50.547841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.534939    4888 fs.go:135] Filesystem UUIDs: map[598e98f0-75fe-4a3b-8fe2-e196d2e6a95f:/dev/nvme0n1p4 76d2441d-5b02-4d59-908b-ee16480da4b9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 02 21:12:50.548147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.534966    4888 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/58ac098165c8d0d81139430c9110f7fa9f8d380a93ad35cd1527f826a58cc889/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/58ac098165c8d0d81139430c9110f7fa9f8d380a93ad35cd1527f826a58cc889/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/59c2689c6131bf09663f0d1b01142ba0dd19e907b8e747f47d977efe14ebc686/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/59c2689c6131bf09663f0d1b01142ba0dd19e907b8e747f47d977efe14ebc686/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d4c3fd60e6b6e1e4a9fba3bcdb4ecff572a47cbbff200839a39002d8f92b9d38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d4c3fd60e6b6e1e4a9fba3bcdb4ecff572a47cbbff200839a39002d8f92b9d38/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d89ca3fe8c57fb8ddc57c483e19f886cac048145dcf4cc81c8cf3cb995c15e29/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d89ca3fe8c57fb8ddc57c483e19f886cac048145dcf4cc81c8cf3cb995c15e29/userdata/shm major:0 minor:269 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/000770db-14e0-4f8e-aa43-2580970caff3/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/000770db-14e0-4f8e-aa43-2580970caff3/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a750444-34a4-42f7-9262-260e8fe2cd5a/volumes/kubernetes.io~projected/kube-api-access-8z4qr:{mountpoint:/var/lib/kubelet/pods/0a750444-34a4-42f7-9262-260e8fe2cd5a/volumes/kubernetes.io~projected/kube-api-access-8z4qr major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f074bad-307f-4832-8df7-b239864787c3/volumes/kubernetes.io~projected/kube-api-access-jz2ct:{mountpoint:/var/lib/kubelet/pods/0f074bad-307f-4832-8df7-b239864787c3/volumes/kubernetes.io~projected/kube-api-access-jz2ct major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a2829e3-5251-4728-8a16-7efd3bbe5d2c/volumes/kubernetes.io~projected/kube-api-access-rrdjh:{mountpoint:/var/lib/kubelet/pods/1a2829e3-5251-4728-8a16-7efd3bbe5d2c/volumes/kubernetes.io~projected/kube-api-access-rrdjh major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/tmp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~projected/kube-api-access-jfllj:{mountpoint:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~projected/kube-api-access-jfllj major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/kube-api-access-phvpc:{mountpoint:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/kube-api-access-phvpc major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~projected/kube-api-access-llp4k:{mountpoint:/var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~projected/kube-api-access-llp4k major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~secret/cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44199617-59fc-462a-9d0a-bae7f2b73599/volumes/kubernetes.io~projected/kube-api-access-mkgpl:{mountpoint:/var/lib/kubelet/pods/44199617-59fc-462a-9d0a-bae7f2b73599/volumes/kubernetes.io~projected/kube-api-access-mkgpl major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a7c59fb-996f-45b1-8794-da90930d9a03/volumes/kubernetes.io~projected/kube-api-access-gn2lc:{mountpoint:/var/lib/kubelet/pods/4a7c59fb-996f-45b1-8794-da90930d9a03/volumes/kubernetes.io~projected/kube-api-access-gn2lc major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~projected/kube-api-access-dpc9z:{mountpoint:/var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~projected/kube-api-access-dpc9z major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60379e20-430c-4731-8a53-857484f3692a/volumes/kubernetes.io~projected/kube-api-access-mz7dm:{mountpoint:/var/lib/kubelet/pods/60379e20-430c-4731-8a53-857484f3692a/volumes/kubernetes.io~projected/kube-api-access-mz7dm major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa/volumes/kubernetes.io~projected/kube-api-access-mz7q4:{mountpoint:/var/lib/kubelet/pods/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa/volumes/kubernetes.io~projected/kube-api-access-mz7q4 major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/73de7623-757b-4d55-8782-91f9e51c88ee/volumes/kubernetes.io~projected/kube-api-access-kbzwm:{mountpoint:/var/lib/kubelet/pods/73de7623-757b-4d55-8782-91f9e51c88ee/volumes/kubernetes.io~projected/kube-api-access-kbzwm major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79441352-ee32-4ebc-a647-fcd571699795/volumes/kubernetes.io~projected/kube-api-access-kbcvs:{mountpoint:/var/lib/kubelet/pods/79441352-ee32-4ebc-a647-fcd571699795/volumes/kubernetes.io~projected/kube-api-access-kbcvs major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b2d5efa-381e-4af3-ab22-06755fff7cf4/volumes/kubernetes.io~projected/kube-api-access-c68dh:{mountpoint:/var/lib/kubelet/pods/7b2d5efa-381e-4af3-ab22-06755fff7cf4/volumes/kubernetes.io~projected/kube-api-access-c68dh major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~projected/kube-api-access-z7bhb:{mountpoint:/var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~projected/kube-api-access-z7bhb major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~projected/kube-api-access-8qwmr:{mountpoint:/var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~projected/kube-api-access-8qwmr major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/961b46b0-0bad-47e8-ab7d-d91663021330/volumes/kubernetes.io~projected/kube-api-access-pj5vq:{mountpoint:/var/lib/kubelet/pods/961b46b0-0bad-47e8-ab7d-d91663021330/volumes/kubernetes.io~projected/kube-api-access-pj5vq major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~projected/kube-api-access-jk668:{mountpoint:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~projected/kube-api-access-jk668 major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/default-certificate major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/stats-auth major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/baec7585-b77e-43a2-924b-3ebeb54f7406/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/baec7585-b77e-43a2-924b-3ebeb54f7406/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bcae7401-4e6b-446c-827a-74e2ba9eb8e5/volumes/kubernetes.io~projected/kube-api-access-6sm92:{mountpoint:/var/lib/kubelet/pods/bcae7401-4e6b-446c-827a-74e2ba9eb8e5/volumes/kubernetes.io~projected/kube-api-access-6sm92 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~projected/kube-api-access-dd75v:{mountpoint:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~projected/kube-api-access-dd75v major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~projected/kube-api-access-fvq86:{mountpoint:/var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~projected/kube-api-access-fvq86 major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~secret/serving-cert major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3061070-d6da-464b-b0df-56c03513f0da/volumes/kubernetes.io~projected/kube-api-access-5whtm:{mountpoint:/var/lib/kubelet/pods/d3061070-d6da-464b-b0df-56c03513f0da/volumes/kubernetes.io~projected/kube-api-access-5whtm major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83/volumes/kubernetes.io~projected/kube-api-access-6qtzw:{mountpoint:/var/lib/kubelet/pods/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83/volumes/kubernetes.io~projected/kube-api-access-6qtzw major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed6836b7-7312-471d-844b-fa29f3ec0ca2/volumes/kubernetes.io~projected/kube-api-access-mh22r:{mountpoint:/var/lib/kubelet/pods/ed6836b7-7312-471d-844b-fa29f3ec0ca2/volumes/kubernetes.io~projected/kube-api-access-mh22r major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee/volumes/kubernetes.io~projected/kube-api-access-rrrsk:{mountpoint:/var/lib/kubelet/pods/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee/volumes/kubernetes.io~projected/kube-api-access-rrrsk major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/kube-api-access-n8gxn:{mountpoint:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/kube-api-access-n8gxn major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcd7fca4-e943-42d4-8b24-2d765d2adc47/volumes/kubernetes.io~projected/kube-api-access-bwr9h:{mountpoint:/var/lib/kubelet/pods/fcd7fca4-e943-42d4-8b24-2d765d2adc47/volumes/kubernetes.io~projected/kube-api-access-bwr9h major:0 minor:65 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/9d0f1ecb6b890489c93230c705e8290c2b63303f57daa375093f7e55db588d9a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/36e09ed7f9ee9f68a260a96c670432936fafee6f25f66daccabcb7f4992b2e2a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3d3b6330f430f9ed922d702b54fa1b7815de0795c2db12d6ba5bc22370e8ac3e/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/926ecb7dce4c65d367600941c0b76b24d6cec69e87055c032d100e046d2cc7c2/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/b9831ab043fa2f90d76a398510d9b822bde11b7035e6553adcbec97ffb41425a/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-116:{mountpoint:/var/lib/containers/storage/overlay/aa79faf9bf8bf8c1927451c3f209cead7117e2aaa881b7885d826006f3c84367/merged major:0 minor:116 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b21fd18cb5dcf83086b8bec2be7e9eba56e40c0cd6238f342bd958a7b6af54fa/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/28dafb58b878ec8175511ae1937c9c62decb2b94d8ed290e1fc4fe66a7db6902/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/9f5c963dd0ca72aa893c863ca73e525e3fb74fad3ef4c0ee05d5d3c6fed0b4ad/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/956848afd93e307d5aa61ebe2bd6d27bd67c753ae9633a362989f9a2fcce0314/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/069c9a05c38a709e2e044487744572415d94cef253b416859744ef9522154929/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/314529e9181bf84445279831278271502d18a10fc959e4da913fdaed41372da8/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/59983b1c2e0e243162fd31f1331b062282de023cc95a47395de85c93910cf032/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b6b050bc9a3b193748feafe5f89073e35a99855fcf845d9bc4ea1d2ddf485a03/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/33de137d797e757ce8108e79aea8f00ecd011c08d73014a44a92bdd692872c28/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/498aca4487cd489764d9f4c78cffc221c1ce0fa9d646338bdaa3541d8591c1ee/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/f6649fbe9382d53c78507847e879a3e77050112f4a631d00f5a4feba28b0170b/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/646079e5e7df947f0b27b0159d6fdce45aa538aa62470bd135c061aff3f5750f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/5fd61b2ff2aaacd11804cd7da45026be5ab54fe9ff96205ca2a93a8f447494c8/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/012aa1048bec790034445d7da9da9eee9d277e52a14ebded5e63cbf0a50d939e/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/743f55a8251ceec51370c2475ed79cd36625360ed7f5346bd69e170d5f0c6417/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/a120da1e4fdb4cb08e8b22e0b85f22c1d9f4011feca09a17996c3bead819d305/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/44184eb93e8a4fffe6fd29731c53bf1649b9c4704c064a6867bc4122a17ea763/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/51be4ba6ee37e5e3c93b86c00f3dc549b00d78632596bd6e716d5f0edf033cd6/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/909a868ac469280f05a8bf8f8d3a3818bb106cd747062f6a06c3409aea68c9b7/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/396a374a65487eb0ffb49cd25def8abc3c744333fa9805c047df75db6396addc/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/d2b68c47f923e5c5b965fb18e539f31283700b9c3a24d353e7ad86ddcf7cfd45/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/582afe473e00c4b054f30994643951a74e841e33d74c3812a8128949474b8268/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/9d01c404743528151ed7c53ac1216c7bb93c45647e1f105109af3e81f648e5dc/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/23aad3498a29be65b5313ecc7282cb0254416901d84e3728dbd4fbaaf317cfd0/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/f7fd554c30690b0b236a8c3e581fee01ec7f6c37fcdb60aaf2fcbb6ca1c10853/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/8a68419d399f6a9d5e8961775f9d14e16d1a0ef3a0baa871814d343c8d62f418/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/157fc9bc2810b467e40996835363eb4f2e7a98fb8f84e9e6eb8f4be26519a470/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/de91f039873238ebb7ba38eec9c7111d0913b421fd69bfa30c07c6ba77d6341a/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/2320dade45616d463826d6e2d2921a9c6002df48c847cb8780b000f4cf24a0fa/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/b41445b05105f3ada01f609bfcfadaede95c8dac68a3fd0365be84a054a5c3b3/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/c2a32abb265c56c47395fab22ff126f1697f0edcfa771786127a45f82d18b23d/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/fb363f940f595d3df08d82703b340166d9ae6eab0b33278af468292657227c54/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/ac4f82a0b4b4254c303724dccfcb4c731abd4a8df5a174ee09933dde82be12e9/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/2211594f3ff04e8feb77352149c954627f48eefa79718e296847b1ed83b16fef/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/d54c2f554a303875738a4427c5687b66e4a34a9c5a4e37b701d64d461dda19bb/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/cec3c0457ef4394f2b0cf09da26c957e21559f2ab443c1c076b3d822f4edac82/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/851076a381eef54aef2ab508d31ecff5d5884088c11eb060c04753cadbbe7400/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/6d7167cc4ce32057812ad1c17acf48ab0f666fd75c14fa5015fe06ab996b6728/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/c76bd862ea3a611e071c51a6c1f3455ef0d10cca2749002057fffec891178a9d/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/f122cd3804bdf54d23456cc40a727df3cdf2028cb46bb6a5a79b5923da84a82b/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/94469951cc73b30c11cd9d80af31e608c657e4def88963129459feb74de0fcfe/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b218aa0a3906b410921e050a18fb3c6d2dec85573b1a878720d7d0fc71df0b05/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/4083a771a9195eccc62dc64dc1b2157e2d2daee4229f8fe60743dd3d1306d058/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/f64145fbbc760018de9b794356270a688dc0c133977e8381ae98b9eda76fd51f/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/751c85a1537bb6383aeac6df485af157e3e1ca68cb3d4e9c5edc1e500651b7f9/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/08cf114ea3985765f8c797d8a4a1134c8416651f592b7d4853332008b6343907/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/bdd7d31550ad4e076ca7e24797c0df67a63cbb2ed2fc318e10644c08bd0fa31b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/c1aaae58807c021e5770397315f67e0874ec34180ec83d7ae780f668007cab6c/merged major:0 minor:97 fsType:overlay blockSize:0}]
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.545879    4888 manager.go:217] Machine: {Timestamp:2026-11-02 21:12:50.545033967 +0000 UTC m=+0.088825891 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076978 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21c8cd27bb81d8b633d57d6cfc9543 SystemUUID:ec21c8cd-27bb-81d8-b633-d57d6cfc9543 BootID:5568069f-7726-4fb1-aed1-38dc23dd47a8 Filesystems:[{Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~projected/kube-api-access-z7bhb DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~projected/kube-api-access-dpc9z DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/58ac098165c8d0d81139430c9110f7fa9f8d380a93ad35cd1527f826a58cc889/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77b3e73db0ec373069f820ff66771a8aca54aa8e650d7351594d8afd95794fd6/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~projected/kube-api-access-fvq86 DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~projected/kube-api-access-llp4k DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1a2829e3-5251-4728-8a16-7efd3bbe5d2c/volumes/kubernetes.io~projected/kube-api-access-rrdjh DeviceMajor:0 DeviceMinor:222 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ed6836b7-7312-471d-844b-fa29f3ec0ca2/volumes/kubernetes.io~projected/kube-api-access-mh22r DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fcd7fca4-e943-42d4-8b24-2d765d2adc47/volumes/kubernetes.io~projected/kube-api-access-bwr9h DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/126ea3ef5eb78d1d4543fa0b287aa0e4bc4c1d162178fcd967f0085c7e920c06/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee/volumes/kubernetes.io~projected/kube-api-access-rrrsk DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a7c59fb-996f-45b1-8794-da90930d9a03/volumes/kubernetes.io~projected/kube-api-access-gn2lc DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/013b00ce554e56a665e67f66ecc31e184dc983d3f5d5094c01df4b6108ec2be7/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f8b558c2b0fe0cf4af1b23c4256800bb39f581d688acde5e64798edc65072b4/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/0f074bad-307f-4832-8df7-b239864787c3/volumes/kubernetes.io~projected/kube-api-access-jz2ct DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/60379e20-430c-4731-8a53-857484f3692a/volumes/kubernetes.io~projected/kube-api-access-mz7dm DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a78642cb81e20d1d015f46f963d67c6b221110b0d9e5f5b0cac4b2124436495c/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/baec7585-b77e-43a2-924b-3ebeb54f7406/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:268 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b5dd768a574273519bcbebbd229ddcee38756bde9d2df2844b640229f6db39d5/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b2d5efa-381e-4af3-ab22-06755fff7cf4/volumes/kubernetes.io~projected/kube-api-access-c68dh DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/73de7623-757b-4d55-8782-91f9e51c88ee/volumes/kubernetes.io~projected/kube-api-access-kbzwm DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~projected/kube-api-access-dd75v DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~projected/kube-api-access-jk668 DeviceMajor:0 DeviceMinor:223 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c0c24bb4ebd314ccaa09e4452df6d0eb32778102a5ee5ac80bf35d6f923604f0/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-116 DeviceMajor:0 DeviceMinor:116 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~projected/kube-api-access-8qwmr DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/kube-api-access-n8gxn DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/000770db-14e0-4f8e-aa43-2580970caff3/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/327e004791764e5df6bbd43c814ec6b621b5d7da7efe7baeb6548ff3e435fc75/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d2461f0193a08d28fc09d8231a0912a31a8dd234666485037771bd2922add077/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83/volumes/kubernetes.io~projected/kube-api-access-6qtzw DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7b91894b-7a38-4d8f-be77-d76177da71bb/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbea11f1-0e1c-4cdd-be0e-c33538c05fee/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~projected/kube-api-access-jfllj DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c66521af30ef7c0cf833c89df46fbdf865295c118d528e722f40f3461f3da5c0/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/747a8db856702d163731373d8f0f357c87f8a54dbe44499ffcdf7bec95bfeca6/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/44199617-59fc-462a-9d0a-bae7f2b73599/volumes/kubernetes.io~projected/kube-api-access-mkgpl DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3cc1883d-1d29-4185-9ea6-fda6b9e021d1/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a5cc6793-336a-4a83-b367-65858682f31b/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3bfff45598eef773d7143f43f2aef132c760ace2d164830733ccaab10f22f38/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/745fb89c079f3084a4eb7f10426e4951c1c8c68c1e1efe6aefbbe4a5e3a70ffc/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa/volumes/kubernetes.io~projected/kube-api-access-mz7q4 DeviceMajor:0 DeviceMinor:327 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a750444-34a4-42f7-9262-260e8fe2cd5a/volumes/kubernetes.io~projected/kube-api-access-8z4qr DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/961b46b0-0bad-47e8-ab7d-d91663021330/volumes/kubernetes.io~projected/kube-api-access-pj5vq DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d3061070-d6da-464b-b0df-56c03513f0da/volumes/kubernetes.io~projected/kube-api-access-5whtm DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/516a9d1c-5f96-4afb-9b1c-d9cf56e56703/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes/kubernetes.io~projected/kube-api-access-phvpc DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/59c2689c6131bf09663f0d1b01142ba0dd19e907b8e747f47d977efe14ebc686/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d4c3fd60e6b6e1e4a9fba3bcdb4ecff572a47cbbff200839a39002d8f92b9d38/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ace7ba346f8f94c7b5ef239b1bcfb7654e719936a26a206d6a905f0f788042ea/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24687f35807557492a1ea466dde7fc7066e866e88244084c6b0364c4372d400a/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/79441352-ee32-4ebc-a647-fcd571699795/volumes/kubernetes.io~projected/kube-api-access-kbcvs DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c3b18bd9-5d6f-478c-a3ab-164d45979936/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bcae7401-4e6b-446c-827a-74e2ba9eb8e5/volumes/kubernetes.io~projected/kube-api-access-6sm92 DeviceMajor:0 DeviceMinor:289 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/29dc30e2-0650-46df-8112-6a881db4dc3f/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9b717065a34619626913f26299e551dd89eb35a94c616057fc525299e07be187/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d89ca3fe8c57fb8ddc57c483e19f886cac048145dcf4cc81c8cf3cb995c15e29/userdata/shm DeviceMajor:0 DeviceMinor:269 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:013b00ce554e56a MacAddress:4e:34:81:92:21:91 Speed:10000 Mtu:8901} {Name:0f8b558c2b0fe0c MacAddress:6a:3b:e6:02:89:37 Speed:10000 Mtu:8901} {Name:126ea3ef5eb78d1 MacAddress:72:18:f3:6b:92:fb Speed:10000 Mtu:8901} {Name:327e004791764e5 MacAddress:7a:94:97:1a:86:ac Speed:10000 Mtu:8901} {Name:58ac098165c8d0d MacAddress:de:6e:9a:42:d4:a4 Speed:10000 Mtu:8901} {Name:59c2689c6131bf0 MacAddress:36:0d:6f:0a:00:d7 Speed:10000 Mtu:8901} {Name:747a8db856702d1 MacAddress:d2:9b:91:52:92:50 Speed:10000 Mtu:8901} {Name:a78642cb81e20d1 MacAddress:f2:7c:d7:5e:e7:2e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:ce:88:06:23 Speed:0 Mtu:9001} {Name:br-int MacAddress:de:34:91:0f:79:ab Speed:0 Mtu:8901} {Name:c0c24bb4ebd314c MacAddress:46:ca:60:f0:2e:c3 Speed:10000 Mtu:8901} {Name:c3bfff45598eef7 MacAddress:16:4d:81:fa:27:b3 Speed:10000 Mtu:8901} {Name:d4c3fd60e6b6e1e MacAddress:46:c2:9d:12:18:5e Speed:10000 Mtu:8901} {Name:d89ca3fe8c57fb8 MacAddress:56:b2:68:d1:0a:7c Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:ce:88:06:23 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:4e:14:57:59:d6:d0 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:c2:31:84:39:8c:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.546518    4888 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.546588    4888 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.546945    4888 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.546970    4888 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-83.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547121    4888 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547128    4888 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547139    4888 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547159    4888 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547178    4888 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547363    4888 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547485    4888 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547520    4888 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547530    4888 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547542    4888 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547551    4888 kubelet.go:405] "Adding apiserver pod source"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.547560    4888 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.548295    4888 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 21:12:50.548480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.548305    4888 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:12:50.549632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.548839    4888 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 21:12:50.552008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.551804    4888 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 21:12:50.552008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.551862    4888 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:12:50.552008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552008    4888 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 21:12:50.552531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552514    4888 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 21:12:50.552564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552548    4888 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 21:12:50.552673 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552663    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 21:12:50.552708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552679    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 21:12:50.552708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552685    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 21:12:50.552708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552692    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 21:12:50.552708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552701    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 21:12:50.552708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552709    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552716    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552721    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552729    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552735    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552755    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552764    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552789    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 21:12:50.552917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.552795    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 21:12:50.556334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.556319    4888 server.go:1263] "Started kubelet"
Nov 02 21:12:50.556448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.556394    4888 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 21:12:50.556498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.556420    4888 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 21:12:50.556542 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.556503    4888 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 21:12:50.556835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.556817    4888 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 21:12:50.556865 ip-10-0-130-83 systemd[1]: Started Kubernetes Kubelet.
Nov 02 21:12:50.558585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.558570    4888 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 21:12:50.560620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.560601    4888 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 21:12:50.560729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.560668    4888 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 21:12:50.560729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.560680    4888 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 21:12:50.560833 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.560755    4888 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 21:12:50.560915 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.560900    4888 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.561063    4888 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.560814    4888 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:07:10 +0000 UTC" deadline="2028-05-25 07:36:43.446185444 +0000 UTC"
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.561377    4888 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13666h23m52.88481428s"
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.562674    4888 factory.go:55] Registering systemd factory
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.562700    4888 factory.go:223] Registration of the systemd container factory successfully
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.563728    4888 factory.go:153] Registering CRI-O factory
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.563745    4888 factory.go:223] Registration of the crio container factory successfully
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.563802    4888 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.563821    4888 factory.go:103] Registering Raw factory
Nov 02 21:12:50.563967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.563837    4888 manager.go:1199] Started watching for new ooms in manager
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564423    4888 manager.go:319] Starting recovery of all containers
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564697    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b91894b-7a38-4d8f-be77-d76177da71bb" volumeName="kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564745    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564762    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="000770db-14e0-4f8e-aa43-2580970caff3" volumeName="kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564778    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29dc30e2-0650-46df-8112-6a881db4dc3f" volumeName="kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564798    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="961b46b0-0bad-47e8-ab7d-d91663021330" volumeName="kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564814    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29dc30e2-0650-46df-8112-6a881db4dc3f" volumeName="kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564838    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" volumeName="kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564872    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" volumeName="kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.564895    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60379e20-430c-4731-8a53-857484f3692a" volumeName="kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565029    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="961b46b0-0bad-47e8-ab7d-d91663021330" volumeName="kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565054    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3b18bd9-5d6f-478c-a3ab-164d45979936" volumeName="kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565097    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbea11f1-0e1c-4cdd-be0e-c33538c05fee" volumeName="kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565111    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed6836b7-7312-471d-844b-fa29f3ec0ca2" volumeName="kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565130    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="baec7585-b77e-43a2-924b-3ebeb54f7406" volumeName="kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565154    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565173    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a7c59fb-996f-45b1-8794-da90930d9a03" volumeName="kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565187    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa" volumeName="kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565207    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbea11f1-0e1c-4cdd-be0e-c33538c05fee" volumeName="kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565221    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee" volumeName="kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565233    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565251    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f074bad-307f-4832-8df7-b239864787c3" volumeName="kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565264    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565323    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bcae7401-4e6b-446c-827a-74e2ba9eb8e5" volumeName="kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565339    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3061070-d6da-464b-b0df-56c03513f0da" volumeName="kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565352    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565371    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565399    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565416    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73de7623-757b-4d55-8782-91f9e51c88ee" volumeName="kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565474    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79441352-ee32-4ebc-a647-fcd571699795" volumeName="kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565494    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b91894b-7a38-4d8f-be77-d76177da71bb" volumeName="kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565508    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565527    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa" volumeName="kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565572    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565626    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" volumeName="kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565645    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5cc6793-336a-4a83-b367-65858682f31b" volumeName="kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565659    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee" volumeName="kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565677    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565689    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73de7623-757b-4d55-8782-91f9e51c88ee" volumeName="kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565702    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b91894b-7a38-4d8f-be77-d76177da71bb" volumeName="kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565745    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db69b56-3f99-4e09-a330-fda21ed67c76" volumeName="kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565758    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3b18bd9-5d6f-478c-a3ab-164d45979936" volumeName="kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565775    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbea11f1-0e1c-4cdd-be0e-c33538c05fee" volumeName="kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565789    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed6836b7-7312-471d-844b-fa29f3ec0ca2" volumeName="kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565804    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29dc30e2-0650-46df-8112-6a881db4dc3f" volumeName="kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565816    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565827    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cc1883d-1d29-4185-9ea6-fda6b9e021d1" volumeName="kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565891    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3b18bd9-5d6f-478c-a3ab-164d45979936" volumeName="kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565908    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565927    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a750444-34a4-42f7-9262-260e8fe2cd5a" volumeName="kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.565943    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566018    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566038    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83" volumeName="kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566053    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed6836b7-7312-471d-844b-fa29f3ec0ca2" volumeName="kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566074    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="000770db-14e0-4f8e-aa43-2580970caff3" volumeName="kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566089    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a2829e3-5251-4728-8a16-7efd3bbe5d2c" volumeName="kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566141    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3b18bd9-5d6f-478c-a3ab-164d45979936" volumeName="kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566182    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60379e20-430c-4731-8a53-857484f3692a" volumeName="kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566195    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566210    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" volumeName="kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566338    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73de7623-757b-4d55-8782-91f9e51c88ee" volumeName="kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566352    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5cc6793-336a-4a83-b367-65858682f31b" volumeName="kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566374    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79441352-ee32-4ebc-a647-fcd571699795" volumeName="kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566389    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b2d5efa-381e-4af3-ab22-06755fff7cf4" volumeName="kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566403    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3b18bd9-5d6f-478c-a3ab-164d45979936" volumeName="kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566424    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" volumeName="kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566438    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a7c59fb-996f-45b1-8794-da90930d9a03" volumeName="kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566457    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b2d5efa-381e-4af3-ab22-06755fff7cf4" volumeName="kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566472    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5cc6793-336a-4a83-b367-65858682f31b" volumeName="kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566491    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566524    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcd7fca4-e943-42d4-8b24-2d765d2adc47" volumeName="kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566541    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cc1883d-1d29-4185-9ea6-fda6b9e021d1" volumeName="kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566561    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44199617-59fc-462a-9d0a-bae7f2b73599" volumeName="kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566617    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566637    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" volumeName="kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566705    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a7c59fb-996f-45b1-8794-da90930d9a03" volumeName="kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566720    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" volumeName="kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566742    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed6836b7-7312-471d-844b-fa29f3ec0ca2" volumeName="kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566755    4888 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 21:12:50.568693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.566763    4888 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 21:12:50.582201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.582176    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:50.582201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.582191    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:50.590574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.590538    4888 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 21:12:50.592221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.592186    4888 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 21:12:50.592221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.592216    4888 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 21:12:50.593438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.592239    4888 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 21:12:50.593438 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.592285    4888 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 21:12:50.611331 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.611297    4888 generic.go:356] "Generic (PLEG): container finished" podID="92be8601b1e73fbd81a5b1380a217ee7" containerID="17b086dc806a8b4566bac1127cf00d7060e42c866da2c3cc3323fb00ba9599d8" exitCode=0
Nov 02 21:12:50.627700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.627670    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:50.627861 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.627713    4888 generic.go:356] "Generic (PLEG): container finished" podID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a" exitCode=255
Nov 02 21:12:50.644386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644353    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="f35cc76ca9165e964fcbfefe4b6d385488d5b4645e214e043853b8fba76ea778" exitCode=0
Nov 02 21:12:50.644386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644380    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="28a284d2e7833bb0a918836796669ddfb14bef771c3ddd1e86165f37f0386b16" exitCode=0
Nov 02 21:12:50.644386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644391    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="2551233e037a4eb20c41cf329f76ededcacfcf739aea7b6cf6b8ec8da6d72132" exitCode=0
Nov 02 21:12:50.644386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644398    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="5738ae3f98e4d92fc609365ee0436e08f40d2db211a452e16e58ae8a09ac00c4" exitCode=0
Nov 02 21:12:50.644639 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644403    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="b3460a889317646bec91c4bc8e84e84d5b920fb989a884c248c5cd5fc559f4cc" exitCode=0
Nov 02 21:12:50.644639 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.644409    4888 generic.go:356] "Generic (PLEG): container finished" podID="ed6836b7-7312-471d-844b-fa29f3ec0ca2" containerID="07b26b7123bedc61f7464cfe752dee60cb79c1da002b0231b4a06304de1e6115" exitCode=0
Nov 02 21:12:50.656434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.656405    4888 manager.go:324] Recovery completed
Nov 02 21:12:50.661047 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.661022    4888 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:50.675101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.675075    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.678835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.678809    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.678948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.678858    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.678948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.678870    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.681521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681502    4888 cpu_manager.go:226] "Starting" policy="none"
Nov 02 21:12:50.681576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681521    4888 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 21:12:50.681576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681544    4888 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 21:12:50.681576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681522    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.681751 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681734    4888 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 21:12:50.681787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681753    4888 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 21:12:50.681787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681779    4888 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 21:12:50.681787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681785    4888 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 21:12:50.681888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681791    4888 policy_none.go:50] "Start"
Nov 02 21:12:50.681888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681800    4888 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 21:12:50.681888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681811    4888 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 21:12:50.681982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681969    4888 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 21:12:50.682011 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681983    4888 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 21:12:50.682011 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.681989    4888 policy_none.go:44] "Start"
Nov 02 21:12:50.682689 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.682667    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.682728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.682704    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.682728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.682714    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.686308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.686284    4888 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 21:12:50.689382 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689369    4888 manager.go:341] "Starting Device Plugin manager"
Nov 02 21:12:50.689453 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.689400    4888 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 21:12:50.689453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689412    4888 server.go:87] "Starting device plugin registration server"
Nov 02 21:12:50.689675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689662    4888 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 21:12:50.689737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689674    4888 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 21:12:50.689788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689750    4888 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 21:12:50.689951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689872    4888 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 21:12:50.689951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689919    4888 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 21:12:50.689951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689925    4888 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 21:12:50.690129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.689992    4888 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T21:12:50.689902974Z","UUID":"aa7fb8ec-818a-4c0a-bf0e-15bd54e709d7","Handler":null,"Name":"","Endpoint":""}
Nov 02 21:12:50.692272 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.691958    4888 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 21:12:50.692272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.692008    4888 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 21:12:50.692272 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.692043    4888 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-83.us-west-2.compute.internal\" not found"
Nov 02 21:12:50.692272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.692061    4888 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 21:12:50.692819 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.692726    4888 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"]
Nov 02 21:12:50.692913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.692901    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.694998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.694975    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.695091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.695012    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.695135 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.695096    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.695786 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.695761    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.695960 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.695925    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.696784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696765    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.696784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696777    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.696939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696796    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.696939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696803    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.696939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696811    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.696939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.696817    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.697075 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.696978    4888 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.697229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697191    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"7a88fcfc7e7a1dfee7d96883c60ca9fa","Type":"ContainerStarted","Data":"498289d121dd3da2a0af07522aeb35da6201a3fc3448a8481a70135b3cd6878b"}
Nov 02 21:12:50.697396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697238    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"7a88fcfc7e7a1dfee7d96883c60ca9fa","Type":"ContainerStarted","Data":"02f29fddc92df7461277fb89c89e7c2708368a82fc3d460b2c229d941a1f62d5"}
Nov 02 21:12:50.697396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697257    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerStarted","Data":"eb53bc9079f188187931209a861c304ef81f64e1b92c6188c4fea41c28e474a3"}
Nov 02 21:12:50.697396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697270    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerDied","Data":"17b086dc806a8b4566bac1127cf00d7060e42c866da2c3cc3323fb00ba9599d8"}
Nov 02 21:12:50.697396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697290    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal" event={"ID":"92be8601b1e73fbd81a5b1380a217ee7","Type":"ContainerStarted","Data":"0f1f752778d0282af9c5495ce3e743ca478ae5b85c56b0761d7ac107721a57e8"}
Nov 02 21:12:50.697396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.697391    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.698397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.698379    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.698494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.698407    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.698494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.698423    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.698593 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.698552    4888 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.us-west-2.compute.internal\" not found" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.746584 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.746555    4888 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 21:12:50.774912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.774800    4888 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:50.789962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.789930    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:12:50.791138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.791119    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:12:50.791211 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.791152    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:12:50.791211 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.791162    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:12:50.791211 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.791204    4888 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.794030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.794013    4888 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:50.794102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.794088    4888 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:50.841108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.841078    4888 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:12:50.847339 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.847318    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.847456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.847345    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.847456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.847365    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.861479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.861450    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.890255 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.890219    4888 nodeinfomanager.go:471] existing CSINode "ip-10-0-130-83.us-west-2.compute.internal" is owned by different node (oldNodeID="963a005a-ca77-41c9-9c8a-3c149a9fb866", newNodeID=""), cleaning up...
Nov 02 21:12:50.895966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.895941    4888 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.896084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.896038    4888 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.896613 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.896597    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeNotReady"
Nov 02 21:12:50.896682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.896625    4888 setters.go:546] "Node became not ready" node="ip-10-0-130-83.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T21:12:50Z","lastTransitionTime":"2026-11-02T21:12:50Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 21:12:50.919296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.919239    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:12:50.919296 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.919288    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.919425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.919303    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.947993    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.948022    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.948039    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.948082    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a88fcfc7e7a1dfee7d96883c60ca9fa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"7a88fcfc7e7a1dfee7d96883c60ca9fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.948103    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.948156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.948104    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92be8601b1e73fbd81a5b1380a217ee7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" (UID: \"92be8601b1e73fbd81a5b1380a217ee7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:50.972093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:50.972065    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:12:50.972217 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:50.972116    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:51.548015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.547980    4888 apiserver.go:52] "Watching apiserver"
Nov 02 21:12:51.646708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.646648    4888 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 21:12:51.647394 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.647366    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-fsnkc","openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8","openshift-dns/dns-default-6trh7","openshift-insights/insights-runtime-extractor-sbzp5","kube-system/kas-connection-checker-685895c7c9-jtg5l","openshift-network-operator/iptables-alerter-7n46x","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs","openshift-console-operator/console-operator-7cd56b8dbc-79wcm","openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw","openshift-network-diagnostics/network-check-target-pg9d6","kube-system/konnectivity-agent-jp6br","kube-system/kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal","openshift-image-registry/image-registry-64b547d4cf-j2p2p","openshift-multus/multus-zmxlz","openshift-image-registry/node-ca-s6ntp","openshift-ingress-canary/ingress-canary-nh249","openshift-insights/insights-operator-7c4986d449-42685","openshift-ingress/router-default-79fb5bb755-mzvm9","openshift-dns/node-resolver-x2blv","openshift-multus/multus-additional-cni-plugins-x7qwl","openshift-image-registry/image-registry-86d4c9b5d-89lds","openshift-cluster-node-tuning-operator/tuned-94n6x","kube-system/kas-connection-checker-685895c7c9-m9fk8","openshift-multus/network-metrics-daemon-wlkwk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr","openshift-network-console/networking-console-plugin-86c478bb88-m9b57","kube-system/global-pull-secret-syncer-pb8vt","openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9","kube-system/kas-connection-checker-685895c7c9-w69f9"]
Nov 02 21:12:51.650336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.648207    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:51.651480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.651462    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.652098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.652081    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.653112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.653094    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.653426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.653409    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.654542 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.654523    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:51.660368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.660349    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:51.661092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.661075    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.662579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.662555    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.663219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.663202    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.704344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.704317    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:12:51.704479 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.704364    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.us-west-2.compute.internal"
Nov 02 21:12:51.751914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.751883    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5vq\" (UniqueName: \"kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.751914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.751916    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.752075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.751943    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.752075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.751968    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzwm\" (UniqueName: \"kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.752156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752114    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.752156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752147    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.752212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752164    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.752212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752183    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.752309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752288    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-tmp\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.752359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752326    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.752429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752414    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.752481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752464    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.752532 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752508    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.752586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752537    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.752586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752566    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.752681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752592    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.752681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752632    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.752681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752665    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:51.752826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752690    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:51.752826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752712    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfllj\" (UniqueName: \"kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.752826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752739    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.752826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752764    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7dm\" (UniqueName: \"kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.752826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752804    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752828    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752931    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752957    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752976    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.752992    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.753056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753014    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753087    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-snapshots\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753095    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753124    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753165    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk668\" (UniqueName: \"kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753198    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753224    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753262    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh22r\" (UniqueName: \"kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.753312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753286    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753317    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753348    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753390    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753424    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753468    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753532    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753556    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-tmp-dir\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753557    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.753603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753591    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753614    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753635    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753651    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753655    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753688    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753713    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753842    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llp4k\" (UniqueName: \"kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753882    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753900    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753917    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:51.753971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753932    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c68dh\" (UniqueName: \"kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.754319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.753978    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.754319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrsk\" (UniqueName: \"kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.754319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754051    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.754319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754079    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.754513 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754487    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.754641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754542    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.754641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754619    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2ct\" (UniqueName: \"kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:51.754759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754657    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.754960    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755038    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whtm\" (UniqueName: \"kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm\") pod \"kas-connection-checker-685895c7c9-jtg5l\" (UID: \"d3061070-d6da-464b-b0df-56c03513f0da\") " pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755077    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755124    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwmr\" (UniqueName: \"kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755160    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755220    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755267    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755301    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755344    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755369    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755412    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755446    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755490    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755518    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755543    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755590    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkgpl\" (UniqueName: \"kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl\") pod \"kas-connection-checker-685895c7c9-m9fk8\" (UID: \"44199617-59fc-462a-9d0a-bae7f2b73599\") " pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755639    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qtzw\" (UniqueName: \"kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw\") pod \"kas-connection-checker-685895c7c9-w69f9\" (UID: \"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83\") " pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755675    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755746    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755776    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755825    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755883    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755910    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd75v\" (UniqueName: \"kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755954    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.755985    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756033    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvq86\" (UniqueName: \"kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756063    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756101    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756135    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756177    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756207    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756252    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756281    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756325    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756358    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwr9h\" (UniqueName: \"kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756402    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756428    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756493    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756530    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756576    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756604    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756649    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756681    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756725    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4qr\" (UniqueName: \"kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr\") pod \"volume-data-source-validator-58c9f5f46d-8nshs\" (UID: \"0a750444-34a4-42f7-9262-260e8fe2cd5a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756754    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756793    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bhb\" (UniqueName: \"kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756824    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756874    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpc9z\" (UniqueName: \"kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756909    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbcvs\" (UniqueName: \"kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756948    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.756979    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757023    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757055    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757098    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757124    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757166    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757199    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757242    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757273    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757325    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757361    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdjh\" (UniqueName: \"kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh\") pod \"network-check-source-6f78d59f99-l4hv8\" (UID: \"1a2829e3-5251-4728-8a16-7efd3bbe5d2c\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757408    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757447    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757492    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757524    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757569    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757600    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757647    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757719    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.758167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.757749    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.760671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.758375    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-tmp\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.760671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.759001    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.760671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.758444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-tuned\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.761600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.761582    4888 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 21:12:51.775268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775234    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 21:12:51.775268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775234    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775235    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775273    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775235    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775297    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775279    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775315    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775330    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775334    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775363    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775389    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775396    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775424    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 21:12:51.775485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775430    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775728    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775733    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775760    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775770    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775764    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775760    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775803    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775805    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 21:12:51.775981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775832    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.775998    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776041    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776066    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776106    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776135    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776174    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776204    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776248    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 21:12:51.776371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776322    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776378    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776407    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776413    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776468    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776479    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bj87x\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776470    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776516    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776525    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776529    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hb5qv\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776538    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vtj4n\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776691    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776705    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-q2w7t\""
Nov 02 21:12:51.776712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776710    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776715    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776737    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zkwqq\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776785    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-crvzm\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776790    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776793    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c5zxb\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776813    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nvsgt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776820    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776821    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c6fz8\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776919    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776953    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4znh\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.776993    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777021    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777036    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777042    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777072    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mhk5v\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777079    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777089    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jzn4w\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777103    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dfd2\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777200    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jzpsf\""
Nov 02 21:12:51.777260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777265    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777330    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777359    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b8fk2\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777408    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777416    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777454    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777458    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777477    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777482    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777484    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777505    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lplw4\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777567    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777620    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777627    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777755    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777788    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ps8jp\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777810    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777828    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777838    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777843    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777830    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777891    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777759    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777905    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-q4vfp\""
Nov 02 21:12:51.777935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.777932    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778006    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zzckx\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778034    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gdtlz\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778059    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zlqgx\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778066    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j2kn9\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778061    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dtndn\""
Nov 02 21:12:51.778669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.778074    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 21:12:51.779183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779164    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovn-node-metrics-cert\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.779250 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779212    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-stats-auth\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.779298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779256    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b91894b-7a38-4d8f-be77-d76177da71bb-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.779298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779266    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60379e20-430c-4731-8a53-857484f3692a-serviceca\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.779298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779284    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.779433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779288    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.779433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779357    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.779433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779408    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9db69b56-3f99-4e09-a330-fda21ed67c76-nginx-conf\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:51.779571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779476    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.779571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779499    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.779571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779516    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.779571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779556    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-serving-cert\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.779708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.779694    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.780576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.780561    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000770db-14e0-4f8e-aa43-2580970caff3-konnectivity-ca\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:51.781732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.781712    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-config\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.781796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.781734    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-config\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.781796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.781718    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-cert\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:51.781796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.781743    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-default-certificate\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.781796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.781785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-cni-binary-copy\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.782803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.782787    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-serving-cert\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.782803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.782795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-env-overrides\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.783987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.783970    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-service-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.784058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.784014    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-config\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.784101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.784066    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/73de7623-757b-4d55-8782-91f9e51c88ee-multus-daemon-config\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.784263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.784249    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000770db-14e0-4f8e-aa43-2580970caff3-agent-certs\") pod \"konnectivity-agent-jp6br\" (UID: \"000770db-14e0-4f8e-aa43-2580970caff3\") " pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:12:51.785039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.785024    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.785116 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.785092    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-serving-cert\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.785168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.785116    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/961b46b0-0bad-47e8-ab7d-d91663021330-iptables-alerter-script\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.785168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.785126    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b91894b-7a38-4d8f-be77-d76177da71bb-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.797841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.797818    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sdlzm\""
Nov 02 21:12:51.797948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.797889    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z2vtc\""
Nov 02 21:12:51.822049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.821984    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mnft8\""
Nov 02 21:12:51.823089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.823067    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 21:12:51.823393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.823375    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 21:12:51.823842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.823825    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 21:12:51.824347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.824332    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 21:12:51.825112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.825097    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3b18bd9-5d6f-478c-a3ab-164d45979936-ovnkube-script-lib\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.825760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.825731    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 21:12:51.825760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.825746    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 21:12:51.825920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.825729    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 21:12:51.826367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826348    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.826367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826358    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2ct\" (UniqueName: \"kubernetes.io/projected/0f074bad-307f-4832-8df7-b239864787c3-kube-api-access-jz2ct\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:51.826483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826389    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh22r\" (UniqueName: \"kubernetes.io/projected/ed6836b7-7312-471d-844b-fa29f3ec0ca2-kube-api-access-mh22r\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.826483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826453    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrsk\" (UniqueName: \"kubernetes.io/projected/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-kube-api-access-rrrsk\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.826588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826534    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk668\" (UniqueName: \"kubernetes.io/projected/a5cc6793-336a-4a83-b367-65858682f31b-kube-api-access-jk668\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.826588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826534    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzwm\" (UniqueName: \"kubernetes.io/projected/73de7623-757b-4d55-8782-91f9e51c88ee-kube-api-access-kbzwm\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.826990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.826967    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7dm\" (UniqueName: \"kubernetes.io/projected/60379e20-430c-4731-8a53-857484f3692a-kube-api-access-mz7dm\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.827059 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827046    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.827112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827091    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sm92\" (UniqueName: \"kubernetes.io/projected/bcae7401-4e6b-446c-827a-74e2ba9eb8e5-kube-api-access-6sm92\") pod \"network-check-target-pg9d6\" (UID: \"bcae7401-4e6b-446c-827a-74e2ba9eb8e5\") " pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:51.827186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827160    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qtzw\" (UniqueName: \"kubernetes.io/projected/e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83-kube-api-access-6qtzw\") pod \"kas-connection-checker-685895c7c9-w69f9\" (UID: \"e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83\") " pod="kube-system/kas-connection-checker-685895c7c9-w69f9"
Nov 02 21:12:51.827238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827189    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c68dh\" (UniqueName: \"kubernetes.io/projected/7b2d5efa-381e-4af3-ab22-06755fff7cf4-kube-api-access-c68dh\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.827492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827459    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.827492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827479    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5vq\" (UniqueName: \"kubernetes.io/projected/961b46b0-0bad-47e8-ab7d-d91663021330-kube-api-access-pj5vq\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.827636 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827532    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whtm\" (UniqueName: \"kubernetes.io/projected/d3061070-d6da-464b-b0df-56c03513f0da-kube-api-access-5whtm\") pod \"kas-connection-checker-685895c7c9-jtg5l\" (UID: \"d3061070-d6da-464b-b0df-56c03513f0da\") " pod="kube-system/kas-connection-checker-685895c7c9-jtg5l"
Nov 02 21:12:51.827636 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827222    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.827636 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.827596    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bhb\" (UniqueName: \"kubernetes.io/projected/7b91894b-7a38-4d8f-be77-d76177da71bb-kube-api-access-z7bhb\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9\" (UID: \"7b91894b-7a38-4d8f-be77-d76177da71bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9"
Nov 02 21:12:51.828045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828021    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdjh\" (UniqueName: \"kubernetes.io/projected/1a2829e3-5251-4728-8a16-7efd3bbe5d2c-kube-api-access-rrdjh\") pod \"network-check-source-6f78d59f99-l4hv8\" (UID: \"1a2829e3-5251-4728-8a16-7efd3bbe5d2c\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-l4hv8"
Nov 02 21:12:51.828113 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828027    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4qr\" (UniqueName: \"kubernetes.io/projected/0a750444-34a4-42f7-9262-260e8fe2cd5a-kube-api-access-8z4qr\") pod \"volume-data-source-validator-58c9f5f46d-8nshs\" (UID: \"0a750444-34a4-42f7-9262-260e8fe2cd5a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-8nshs"
Nov 02 21:12:51.828222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828140    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvq86\" (UniqueName: \"kubernetes.io/projected/cbea11f1-0e1c-4cdd-be0e-c33538c05fee-kube-api-access-fvq86\") pod \"service-ca-operator-544d749c8c-bccqw\" (UID: \"cbea11f1-0e1c-4cdd-be0e-c33538c05fee\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw"
Nov 02 21:12:51.828222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828147    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd75v\" (UniqueName: \"kubernetes.io/projected/c3b18bd9-5d6f-478c-a3ab-164d45979936-kube-api-access-dd75v\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.828222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828149    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwr9h\" (UniqueName: \"kubernetes.io/projected/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kube-api-access-bwr9h\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.828611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828544    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbcvs\" (UniqueName: \"kubernetes.io/projected/79441352-ee32-4ebc-a647-fcd571699795-kube-api-access-kbcvs\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.828735 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828712    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpc9z\" (UniqueName: \"kubernetes.io/projected/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-kube-api-access-dpc9z\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.831876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.828934    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkgpl\" (UniqueName: \"kubernetes.io/projected/44199617-59fc-462a-9d0a-bae7f2b73599-kube-api-access-mkgpl\") pod \"kas-connection-checker-685895c7c9-m9fk8\" (UID: \"44199617-59fc-462a-9d0a-bae7f2b73599\") " pod="kube-system/kas-connection-checker-685895c7c9-m9fk8"
Nov 02 21:12:51.831876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.829113    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfllj\" (UniqueName: \"kubernetes.io/projected/29dc30e2-0650-46df-8112-6a881db4dc3f-kube-api-access-jfllj\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.831876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.829304    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llp4k\" (UniqueName: \"kubernetes.io/projected/3cc1883d-1d29-4185-9ea6-fda6b9e021d1-kube-api-access-llp4k\") pod \"ingress-canary-nh249\" (UID: \"3cc1883d-1d29-4185-9ea6-fda6b9e021d1\") " pod="openshift-ingress-canary/ingress-canary-nh249"
Nov 02 21:12:51.831876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.829341    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwmr\" (UniqueName: \"kubernetes.io/projected/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-kube-api-access-8qwmr\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.831876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.829472    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.832286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.832267    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.832589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.832571    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.832771 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.832758    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baec7585-b77e-43a2-924b-3ebeb54f7406-original-pull-secret\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.834273 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.834246    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-42685\" (UID: \"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd\") " pod="openshift-insights/insights-operator-7c4986d449-42685"
Nov 02 21:12:51.834373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.834310    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/516a9d1c-5f96-4afb-9b1c-d9cf56e56703-trusted-ca\") pod \"console-operator-7cd56b8dbc-79wcm\" (UID: \"516a9d1c-5f96-4afb-9b1c-d9cf56e56703\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:51.834373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.834323    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7b2d5efa-381e-4af3-ab22-06755fff7cf4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.834373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.834344    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.858956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.858919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.858956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.858959    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.858976    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.858993    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859007    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859020    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859035    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859064    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-sys-fs\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859085    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-cnibin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859082    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-netd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859063    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-hostroot\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859063    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859097    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-systemd\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859117    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859136    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-dbus\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859143    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859163    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-ovn\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859162    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-kubelet\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859178    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859204    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859205    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-run\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859222    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859241    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-node-log\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859273    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859303    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859328    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859353    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859367    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-cni-bin\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859359    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-run-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859380    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859395    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee-hosts-file\") pod \"node-resolver-x2blv\" (UID: \"f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee\") " pod="openshift-dns/node-resolver-x2blv"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859403    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859399    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-bin\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859431    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859457    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859435    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-conf\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859487    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-device-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859479    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859510    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859518    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-cni-multus\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859527    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859527    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-run-netns\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859556    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859573    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a7c59fb-996f-45b1-8794-da90930d9a03-tmp-dir\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859580    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859588    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-lib-modules\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859607    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859609    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859630    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-cnibin\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859633    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859658    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859670    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-data-volume\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859635    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-var-lib-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.859686    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.359661286 +0000 UTC m=+1.903453195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859702    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-registration-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859739    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.859748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859760    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7q4\" (UniqueName: \"kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859787    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-multus-certs\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859788    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859793    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-socket-dir-parent\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859823    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859865    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859894    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn2lc\" (UniqueName: \"kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859948    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859973    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.859974    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.859997    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860004    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-systemd-units\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860018    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-etc-kubernetes\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860024    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-netns\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.859975    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860025    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a7c59fb-996f-45b1-8794-da90930d9a03-config-volume\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860064    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-var-lib-kubelet\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860077    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360045942 +0000 UTC m=+1.903837858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860079    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860131    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360114856 +0000 UTC m=+1.903906751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860150    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860175    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860197    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860219    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860223    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-kubernetes\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860227    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/961b46b0-0bad-47e8-ab7d-d91663021330-host-slash\") pod \"iptables-alerter-7n46x\" (UID: \"961b46b0-0bad-47e8-ab7d-d91663021330\") " pod="openshift-network-operator/iptables-alerter-7n46x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860240    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860266    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860284    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860304    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysconfig\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860313    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860265    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcd7fca4-e943-42d4-8b24-2d765d2adc47-socket-dir\") pod \"aws-ebs-csi-driver-node-9kqrr\" (UID: \"fcd7fca4-e943-42d4-8b24-2d765d2adc47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kqrr"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860266    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-slash\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860324    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860306    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860348    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860359    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-var-lib-kubelet\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860368    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360357787 +0000 UTC m=+1.904149684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860404    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860429    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60379e20-430c-4731-8a53-857484f3692a-host\") pod \"node-ca-s6ntp\" (UID: \"60379e20-430c-4731-8a53-857484f3692a\") " pod="openshift-image-registry/node-ca-s6ntp"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860411    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860437    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860474    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360465609 +0000 UTC m=+1.904257508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860475    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860493    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860511    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860526    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860541    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860556    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-multus-conf-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860556    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860577    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-etc-openvswitch\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860586    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860602    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-sys\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860617    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860642    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-modprobe-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860670    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860680    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860702    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860738    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-log-socket\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860753    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860753    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-host-run-k8s-cni-cncf-io\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860769    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360748429 +0000 UTC m=+1.904540326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860772    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed6836b7-7312-471d-844b-fa29f3ec0ca2-system-cni-dir\") pod \"multus-additional-cni-plugins-x7qwl\" (UID: \"ed6836b7-7312-471d-844b-fa29f3ec0ca2\") " pod="openshift-multus/multus-additional-cni-plugins-x7qwl"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860831    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860870    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-sysctl-d\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860885    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860910    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860921    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-etc-systemd\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860930    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baec7585-b77e-43a2-924b-3ebeb54f7406-kubelet-config\") pod \"global-pull-secret-syncer-pb8vt\" (UID: \"baec7585-b77e-43a2-924b-3ebeb54f7406\") " pod="kube-system/global-pull-secret-syncer-pb8vt"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860936    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.860972    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860972    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.860976    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29dc30e2-0650-46df-8112-6a881db4dc3f-host\") pod \"tuned-94n6x\" (UID: \"29dc30e2-0650-46df-8112-6a881db4dc3f\") " pod="openshift-cluster-node-tuning-operator/tuned-94n6x"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.861006    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.360998874 +0000 UTC m=+1.904790794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861030    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861050    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861066    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.861038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861086    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:51.863615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861103    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73de7623-757b-4d55-8782-91f9e51c88ee-system-cni-dir\") pod \"multus-zmxlz\" (UID: \"73de7623-757b-4d55-8782-91f9e51c88ee\") " pod="openshift-multus/multus-zmxlz"
Nov 02 21:12:51.863615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.861126    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b18bd9-5d6f-478c-a3ab-164d45979936-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsnkc\" (UID: \"c3b18bd9-5d6f-478c-a3ab-164d45979936\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:51.863615 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.861176    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:51.863615 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.861224    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.361212198 +0000 UTC m=+1.905004106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:51.913950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.913920    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7q4\" (UniqueName: \"kubernetes.io/projected/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-api-access-mz7q4\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.915279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.915260    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn2lc\" (UniqueName: \"kubernetes.io/projected/4a7c59fb-996f-45b1-8794-da90930d9a03-kube-api-access-gn2lc\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.951887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.951859    4888 scope.go:122] "RemoveContainer" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"
Nov 02 21:12:51.962389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.962365    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.962510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.962405    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:51.962510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.962463    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.962510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.962478    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-crio-socket\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.962510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:51.962494    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962576    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962608    4888 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962636    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.462621195 +0000 UTC m=+2.006413092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962576    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962653    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.462641624 +0000 UTC m=+2.006433537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : configmap "kube-rbac-proxy" not found
Nov 02 21:12:51.962770 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:51.962674    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:52.46266766 +0000 UTC m=+2.006459557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:52.365723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365645    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:52.365723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365687    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:52.365723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365705    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365732    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365753    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365784    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.365805    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365813    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365888    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365909    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365815    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365819    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365947    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365824    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.365804826 +0000 UTC m=+2.909596734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365889    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:52.365973 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365892    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.365999    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.365987407 +0000 UTC m=+2.909779306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366022    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366016379 +0000 UTC m=+2.909808276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366037    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366031604 +0000 UTC m=+2.909823497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366061    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366055984 +0000 UTC m=+2.909847877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366071    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366066224 +0000 UTC m=+2.909858118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366086    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366075934 +0000 UTC m=+2.909867828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.366133    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366202    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:52.366358 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.366237    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.366230131 +0000 UTC m=+2.910022025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:52.466973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.466935    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.467003    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.467035    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467071    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467096    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467128    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.467113434 +0000 UTC m=+3.010905340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467146    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.467134931 +0000 UTC m=+3.010926824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467143    4888 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 21:12:52.467175 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.467180    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:53.467170364 +0000 UTC m=+3.010962257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : configmap "kube-rbac-proxy" not found
Nov 02 21:12:52.652499 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.652422    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/2.log"
Nov 02 21:12:52.652927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.652838    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/1.log"
Nov 02 21:12:52.652927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.652892    4888 generic.go:356] "Generic (PLEG): container finished" podID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f" exitCode=255
Nov 02 21:12:52.653012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.652965    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerDied","Data":"2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"}
Nov 02 21:12:52.653048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.653020    4888 scope.go:122] "RemoveContainer" containerID="e897fe2bf1ace822f11b0d822d1930264f141378f3d96d9fad0ae34b9c16275a"
Nov 02 21:12:52.653463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:52.653442    4888 scope.go:122] "RemoveContainer" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"
Nov 02 21:12:52.653661 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:52.653637    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:12:53.376150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376112    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376167    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376194    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376211    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376239    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376249    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376258    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376311    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376290738 +0000 UTC m=+4.920082635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376333    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376340    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376353    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376367    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376355    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376335192 +0000 UTC m=+4.920127087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376408    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.37639761 +0000 UTC m=+4.920189509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376425    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376415779 +0000 UTC m=+4.920207721 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:53.376437 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376428    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376458    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.37644914 +0000 UTC m=+4.920241033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376457    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376476    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376467185 +0000 UTC m=+4.920259085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376504    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.376506    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376513    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376544    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376532839 +0000 UTC m=+4.920324761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376575    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:53.376957 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.376605    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.376598845 +0000 UTC m=+4.920390739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:53.477695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.477660    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:53.477900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.477718    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:53.477900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.477740    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:53.477900 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.477809    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:53.477900 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.477878    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.477864874 +0000 UTC m=+5.021656781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:53.477900 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.477809    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:53.478177 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:53.477937    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:55.477924657 +0000 UTC m=+5.021716564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:53.478380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.478358    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:53.658472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:53.658445    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/2.log"
Nov 02 21:12:55.038274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.038239    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:12:55.396280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396188    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:55.396280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396246    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:55.396280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396264    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396295    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396319    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396361    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396342155 +0000 UTC m=+8.940134049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396370    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396442    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396453    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396457    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396465    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396370    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396398    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396468    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396447282 +0000 UTC m=+8.940239193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396539    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.39652931 +0000 UTC m=+8.940321207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:55.396541 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396551    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396545532 +0000 UTC m=+8.940337426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396573    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396563463 +0000 UTC m=+8.940355357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396597    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396604    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396588185 +0000 UTC m=+8.940380088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396652    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.396701    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396743    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396768    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396761549 +0000 UTC m=+8.940553489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396811    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.396885    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.396839223 +0000 UTC m=+8.940631119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.397006    4888 scope.go:122] "RemoveContainer" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"
Nov 02 21:12:55.397238 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.397201    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:12:55.497118 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.497081    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:55.497298 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.497240    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:55.497626 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.497593    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.497559518 +0000 UTC m=+9.041351435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:55.497785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.497713    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:55.497895 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.497804    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 21:12:55.497895 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:55.497882    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls podName:4a7c59fb-996f-45b1-8794-da90930d9a03 nodeName:}" failed. No retries permitted until 2026-11-02 21:12:59.497842368 +0000 UTC m=+9.041634273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls") pod "dns-default-6trh7" (UID: "4a7c59fb-996f-45b1-8794-da90930d9a03") : secret "dns-default-metrics-tls" not found
Nov 02 21:12:55.526706 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:55.526671    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:12:56.633396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:56.633369    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x2blv_f32f0bd3-2a4b-4ff0-a531-b5c19fd5c1ee/dns-node-resolver/0.log"
Nov 02 21:12:57.635136 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:57.635108    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s6ntp_60379e20-430c-4731-8a53-857484f3692a/node-ca/0.log"
Nov 02 21:12:58.234971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:58.234940    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nh249_3cc1883d-1d29-4185-9ea6-fda6b9e021d1/serve-healthcheck-canary/0.log"
Nov 02 21:12:59.039615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.039584    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9_7b91894b-7a38-4d8f-be77-d76177da71bb/kube-storage-version-migrator-operator/0.log"
Nov 02 21:12:59.437833 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.437790    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.437862    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.437914    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.437961    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.437969    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.437978    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86d4c9b5d-89lds: secret "image-registry-tls" not found
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.438010    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:12:59.438035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.438037    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438015    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438090    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438098    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438101    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438056    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls podName:3c91bfcd-ce7c-4910-a76f-6d018ac11daa nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.43803553 +0000 UTC m=+16.981827443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls") pod "image-registry-86d4c9b5d-89lds" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa") : secret "image-registry-tls" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438230    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls podName:7b2d5efa-381e-4af3-ab22-06755fff7cf4 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.438201217 +0000 UTC m=+16.981993115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-p8cfn" (UID: "7b2d5efa-381e-4af3-ab22-06755fff7cf4") : secret "cluster-monitoring-operator-tls" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438250    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.438240109 +0000 UTC m=+16.982032007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438273    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs podName:0f074bad-307f-4832-8df7-b239864787c3 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.43826431 +0000 UTC m=+16.982056206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs") pod "network-metrics-daemon-wlkwk" (UID: "0f074bad-307f-4832-8df7-b239864787c3") : secret "metrics-daemon-secret" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438295    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.438287552 +0000 UTC m=+16.982079448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : secret "router-metrics-certs-default" not found
Nov 02 21:12:59.438324 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438308    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls podName:79441352-ee32-4ebc-a647-fcd571699795 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.43830236 +0000 UTC m=+16.982094254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-ss2jw" (UID: "79441352-ee32-4ebc-a647-fcd571699795") : secret "samples-operator-tls" not found
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.438351    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.438381    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438485    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438491    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438503    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b547d4cf-j2p2p: secret "image-registry-tls" not found
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438531    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert podName:9db69b56-3f99-4e09-a330-fda21ed67c76 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.438520162 +0000 UTC m=+16.982312073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-m9b57" (UID: "9db69b56-3f99-4e09-a330-fda21ed67c76") : secret "networking-console-plugin-cert" not found
Nov 02 21:12:59.438632 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.438544    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls podName:f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83 nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.438539132 +0000 UTC m=+16.982331026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls") pod "image-registry-64b547d4cf-j2p2p" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83") : secret "image-registry-tls" not found
Nov 02 21:12:59.539261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.539211    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:59.539448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.539297    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:12:59.539448 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.539388    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:59.539535 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:12:59.539455    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls podName:6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa nodeName:}" failed. No retries permitted until 2026-11-02 21:13:07.539436994 +0000 UTC m=+17.083228890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbzp5" (UID: "6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa") : secret "insights-runtime-extractor-tls" not found
Nov 02 21:12:59.539665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.539647    4888 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 21:12:59.541659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.541642    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7c59fb-996f-45b1-8794-da90930d9a03-metrics-tls\") pod \"dns-default-6trh7\" (UID: \"4a7c59fb-996f-45b1-8794-da90930d9a03\") " pod="openshift-dns/dns-default-6trh7"
Nov 02 21:12:59.763550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:12:59.763470    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6trh7"
Nov 02 21:13:00.124301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.124226    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6trh7"]
Nov 02 21:13:00.127475 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:00.127444    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7c59fb_996f_45b1_8794_da90930d9a03.slice/crio-b3d82a10d1cabed3304584e03bb968bb8b1be2efd686886699ba4a02ac4b70b5 WatchSource:0}: Error finding container b3d82a10d1cabed3304584e03bb968bb8b1be2efd686886699ba4a02ac4b70b5: Status 404 returned error can't find the container with id b3d82a10d1cabed3304584e03bb968bb8b1be2efd686886699ba4a02ac4b70b5
Nov 02 21:13:00.129447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.129432    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:13:00.283684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.283657    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:13:00.283868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.283838    4888 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 21:13:00.681156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.681120    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6trh7" event={"ID":"4a7c59fb-996f-45b1-8794-da90930d9a03","Type":"ContainerStarted","Data":"b3d82a10d1cabed3304584e03bb968bb8b1be2efd686886699ba4a02ac4b70b5"}
Nov 02 21:13:00.754016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.753982    4888 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:13:00.754372 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:00.754358    4888 scope.go:122] "RemoveContainer" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"
Nov 02 21:13:00.754543 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:00.754525    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:13:01.041377 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:01.041298    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-83.us-west-2.compute.internal" event="NodeReady"
Nov 02 21:13:02.593189 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.593160    4888 scope.go:122] "RemoveContainer" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"
Nov 02 21:13:02.691243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.691146    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6trh7" event={"ID":"4a7c59fb-996f-45b1-8794-da90930d9a03","Type":"ContainerStarted","Data":"1728d11eda9ddc8a88f63c21e9e9bb0e37657ebba6d08ee3efd691f32907de30"}
Nov 02 21:13:02.691243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.691219    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6trh7" event={"ID":"4a7c59fb-996f-45b1-8794-da90930d9a03","Type":"ContainerStarted","Data":"baa825587526bac1c8cfe2667930f1632d28ab162897ba7d6792a569a42f5d82"}
Nov 02 21:13:02.691891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.691821    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6trh7"
Nov 02 21:13:02.693946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.693927    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/2.log"
Nov 02 21:13:02.694045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.693988    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerStarted","Data":"2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"}
Nov 02 21:13:02.694260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.694240    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:13:02.696142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.695787    4888 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-79wcm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.21:8443/readyz\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body=
Nov 02 21:13:02.696142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.695836    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.21:8443/readyz\": dial tcp 10.132.0.21:8443: connect: connection refused"
Nov 02 21:13:02.916068 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:02.916038    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsnkc"
Nov 02 21:13:03.698859 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.698818    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:13:03.699438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.699254    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/2.log"
Nov 02 21:13:03.699438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.699342    4888 generic.go:356] "Generic (PLEG): container finished" podID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerID="2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f" exitCode=255
Nov 02 21:13:03.699438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.699371    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerDied","Data":"2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"}
Nov 02 21:13:03.699438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.699419    4888 scope.go:122] "RemoveContainer" containerID="2682a91966bb0e592094f9751e30f29c72559b0d765e4c4db31615185aeed81f"
Nov 02 21:13:03.699648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:03.699629    4888 scope.go:122] "RemoveContainer" containerID="2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"
Nov 02 21:13:03.699867 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:03.699828    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:13:04.704342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:04.704314    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:13:04.704730 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:04.704626    4888 scope.go:122] "RemoveContainer" containerID="2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"
Nov 02 21:13:04.704818 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:04.704800    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:13:07.513961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.513920    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:07.513961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.513970    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.513988    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.514016    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.514043    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.514067    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:07.514097    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle podName:a5cc6793-336a-4a83-b367-65858682f31b nodeName:}" failed. No retries permitted until 2026-11-02 21:13:23.514078588 +0000 UTC m=+33.057870482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle") pod "router-default-79fb5bb755-mzvm9" (UID: "a5cc6793-336a-4a83-b367-65858682f31b") : configmap references non-existent config key: service-ca.crt
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.514131    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:13:07.514563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.514187    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:07.516843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.516809    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b2d5efa-381e-4af3-ab22-06755fff7cf4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-p8cfn\" (UID: \"7b2d5efa-381e-4af3-ab22-06755fff7cf4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:13:07.516986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.516947    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f074bad-307f-4832-8df7-b239864787c3-metrics-certs\") pod \"network-metrics-daemon-wlkwk\" (UID: \"0f074bad-307f-4832-8df7-b239864787c3\") " pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:13:07.517337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.517315    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"image-registry-86d4c9b5d-89lds\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") " pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:07.517408 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.517376    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79441352-ee32-4ebc-a647-fcd571699795-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-ss2jw\" (UID: \"79441352-ee32-4ebc-a647-fcd571699795\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:13:07.517447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.517414    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"image-registry-64b547d4cf-j2p2p\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") " pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:07.517480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.517414    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9db69b56-3f99-4e09-a330-fda21ed67c76-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-m9b57\" (UID: \"9db69b56-3f99-4e09-a330-fda21ed67c76\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:13:07.517761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.517744    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc6793-336a-4a83-b367-65858682f31b-metrics-certs\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:07.552163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.552128    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:07.553186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.553167    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"
Nov 02 21:13:07.554267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.554251    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"
Nov 02 21:13:07.555297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.555286    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57"
Nov 02 21:13:07.560667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.560574    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wlkwk"
Nov 02 21:13:07.561681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.561660    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:07.632675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.632146    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:13:07.634616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.634563    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbzp5\" (UID: \"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa\") " pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:13:07.863590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.863514    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sbzp5"
Nov 02 21:13:07.955330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:07.955297    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn"]
Nov 02 21:13:07.976117 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:07.976082    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2d5efa_381e_4af3_ab22_06755fff7cf4.slice/crio-e35c0b6c350bdcbaf1daa78eec4c0ba39d77b5cb6bbad2dd933e829a19f4c986 WatchSource:0}: Error finding container e35c0b6c350bdcbaf1daa78eec4c0ba39d77b5cb6bbad2dd933e829a19f4c986: Status 404 returned error can't find the container with id e35c0b6c350bdcbaf1daa78eec4c0ba39d77b5cb6bbad2dd933e829a19f4c986
Nov 02 21:13:08.002940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.002909    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wlkwk"]
Nov 02 21:13:08.003354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.003334    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:13:08.021566 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:08.021524    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c91bfcd_ce7c_4910_a76f_6d018ac11daa.slice/crio-1a1b808bef67d03cdded99a49b4787cb6cb5c30fe5af53eb45239a3e455dd29a WatchSource:0}: Error finding container 1a1b808bef67d03cdded99a49b4787cb6cb5c30fe5af53eb45239a3e455dd29a: Status 404 returned error can't find the container with id 1a1b808bef67d03cdded99a49b4787cb6cb5c30fe5af53eb45239a3e455dd29a
Nov 02 21:13:08.022148 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:08.022052    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f074bad_307f_4832_8df7_b239864787c3.slice/crio-045310f1886b585fbbea4f4032fce5f3be1acc2fdd636ea8908372c57a60d889 WatchSource:0}: Error finding container 045310f1886b585fbbea4f4032fce5f3be1acc2fdd636ea8908372c57a60d889: Status 404 returned error can't find the container with id 045310f1886b585fbbea4f4032fce5f3be1acc2fdd636ea8908372c57a60d889
Nov 02 21:13:08.028645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.028625    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-m9b57"]
Nov 02 21:13:08.030750 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:08.030724    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db69b56_3f99_4e09_a330_fda21ed67c76.slice/crio-f651e246621f235137d5f7f6b7015dae83bdebf7af1c5d87641679ffed89425b WatchSource:0}: Error finding container f651e246621f235137d5f7f6b7015dae83bdebf7af1c5d87641679ffed89425b: Status 404 returned error can't find the container with id f651e246621f235137d5f7f6b7015dae83bdebf7af1c5d87641679ffed89425b
Nov 02 21:13:08.033604 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:08.033581    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36b69cc_b9fa_410b_a0f8_7dc6cc2e3c83.slice/crio-e82a9b6c2b0cec679bc4da12e8ba0f1e789784acff7624733cd646bfb71b8475 WatchSource:0}: Error finding container e82a9b6c2b0cec679bc4da12e8ba0f1e789784acff7624733cd646bfb71b8475: Status 404 returned error can't find the container with id e82a9b6c2b0cec679bc4da12e8ba0f1e789784acff7624733cd646bfb71b8475
Nov 02 21:13:08.050704 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.050664    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw"]
Nov 02 21:13:08.051291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.051270    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:13:08.225084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.225060    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sbzp5"]
Nov 02 21:13:08.228475 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:08.228444    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef9d8e0_bc6c_4e33_a1ca_0a2baec0c5fa.slice/crio-ef2d10cb5d112f882b39e0b340643ca50adbcca497d76e0594b3c878733a28e4 WatchSource:0}: Error finding container ef2d10cb5d112f882b39e0b340643ca50adbcca497d76e0594b3c878733a28e4: Status 404 returned error can't find the container with id ef2d10cb5d112f882b39e0b340643ca50adbcca497d76e0594b3c878733a28e4
Nov 02 21:13:08.719971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.719933    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p" event={"ID":"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83","Type":"ContainerStarted","Data":"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"}
Nov 02 21:13:08.720422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.719986    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:08.720422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.720000    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p" event={"ID":"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83","Type":"ContainerStarted","Data":"e82a9b6c2b0cec679bc4da12e8ba0f1e789784acff7624733cd646bfb71b8475"}
Nov 02 21:13:08.721660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.721633    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw" event={"ID":"79441352-ee32-4ebc-a647-fcd571699795","Type":"ContainerStarted","Data":"fa4a542be965de53cb456a52f58f9d4ba01b5bcffccebc177a835ca56d708912"}
Nov 02 21:13:08.722767 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.722743    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn" event={"ID":"7b2d5efa-381e-4af3-ab22-06755fff7cf4","Type":"ContainerStarted","Data":"e35c0b6c350bdcbaf1daa78eec4c0ba39d77b5cb6bbad2dd933e829a19f4c986"}
Nov 02 21:13:08.723818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.723792    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57" event={"ID":"9db69b56-3f99-4e09-a330-fda21ed67c76","Type":"ContainerStarted","Data":"f651e246621f235137d5f7f6b7015dae83bdebf7af1c5d87641679ffed89425b"}
Nov 02 21:13:08.725097 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.725076    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbzp5" event={"ID":"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa","Type":"ContainerStarted","Data":"3c4d63901af3b634d249ed32d4bacb99d578c7eb95e879cb609dd5c691fd764e"}
Nov 02 21:13:08.725187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.725123    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbzp5" event={"ID":"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa","Type":"ContainerStarted","Data":"ef2d10cb5d112f882b39e0b340643ca50adbcca497d76e0594b3c878733a28e4"}
Nov 02 21:13:08.726284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.726260    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wlkwk" event={"ID":"0f074bad-307f-4832-8df7-b239864787c3","Type":"ContainerStarted","Data":"045310f1886b585fbbea4f4032fce5f3be1acc2fdd636ea8908372c57a60d889"}
Nov 02 21:13:08.727718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.727700    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds" event={"ID":"3c91bfcd-ce7c-4910-a76f-6d018ac11daa","Type":"ContainerStarted","Data":"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"}
Nov 02 21:13:08.727800 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.727734    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds" event={"ID":"3c91bfcd-ce7c-4910-a76f-6d018ac11daa","Type":"ContainerStarted","Data":"1a1b808bef67d03cdded99a49b4787cb6cb5c30fe5af53eb45239a3e455dd29a"}
Nov 02 21:13:08.727881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:08.727862    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:09.736049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:09.736017    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbzp5" event={"ID":"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa","Type":"ContainerStarted","Data":"c731a745814ab4e85483692f4a8dcaf3e46f75df8685df3439beb35cf8e4eb2c"}
Nov 02 21:13:10.551584 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.551493    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:13:10.739842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.739796    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-p8cfn" event={"ID":"7b2d5efa-381e-4af3-ab22-06755fff7cf4","Type":"ContainerStarted","Data":"0ea9b0d0887cf2b8ebae97b3bf14b65d77e0156d76e28fee17e4da14d6b18f5b"}
Nov 02 21:13:10.742378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.742346    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-m9b57" event={"ID":"9db69b56-3f99-4e09-a330-fda21ed67c76","Type":"ContainerStarted","Data":"791a2b08a1a3460c6304fb3bcdd056272abc3431dbe9ce4e82de451399b0ec87"}
Nov 02 21:13:10.744505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.744472    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wlkwk" event={"ID":"0f074bad-307f-4832-8df7-b239864787c3","Type":"ContainerStarted","Data":"d744df0f32c17450541a869b6793b4714272ecb94b8d3e5a7f8f859849d9139c"}
Nov 02 21:13:10.744609 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.744534    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wlkwk" event={"ID":"0f074bad-307f-4832-8df7-b239864787c3","Type":"ContainerStarted","Data":"3720f02005e6d7113c3a796f8772207878940639809587b65f7f05b50863e0f8"}
Nov 02 21:13:10.754039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.754007    4888 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:13:10.755843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:10.754517    4888 scope.go:122] "RemoveContainer" containerID="2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"
Nov 02 21:13:10.755843 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:10.754801    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-79wcm_openshift-console-operator(516a9d1c-5f96-4afb-9b1c-d9cf56e56703)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703"
Nov 02 21:13:11.749754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:11.749712    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw" event={"ID":"79441352-ee32-4ebc-a647-fcd571699795","Type":"ContainerStarted","Data":"d40b44607fbeccd5a0d0fb907010c6608d77a55276d7fd754039c4c41b178499"}
Nov 02 21:13:11.750214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:11.749816    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-ss2jw" event={"ID":"79441352-ee32-4ebc-a647-fcd571699795","Type":"ContainerStarted","Data":"320735ec35624fc27b7da3a1e72d8f49613a1e0ea9b8969af8b87fdf1bdac500"}
Nov 02 21:13:12.755399 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:12.755359    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbzp5" event={"ID":"6ef9d8e0-bc6c-4e33-a1ca-0a2baec0c5fa","Type":"ContainerStarted","Data":"90e2349390f5110f05216ce12574284d77faf6cee5f9aa914e0f6cb8b6198393"}
Nov 02 21:13:13.707437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:13.707403    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6trh7"
Nov 02 21:13:21.605393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:21.605361    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jp6br"
Nov 02 21:13:22.259071 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:22.259034    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:13:23.023678 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.023643    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mxv6r"]
Nov 02 21:13:23.070337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.070297    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.167718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.167677    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 21:13:23.167718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.167691    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 21:13:23.167718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.167716    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 21:13:23.167718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.167724    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hbtfx\""
Nov 02 21:13:23.168029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.167785    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 21:13:23.173160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173135    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173173    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-sys\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173281    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-tls\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173308    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-root\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173327    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-wtmp\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173346    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-metrics-client-ca\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173452    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-textfile\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173479    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdrlb\" (UniqueName: \"kubernetes.io/projected/241d2e2a-901b-4e8f-8c85-ad365914e909-kube-api-access-hdrlb\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.173549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.173501    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274540    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274592    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274627    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-sys\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274677    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-tls\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274705    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-root\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274730    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-wtmp\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274752    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-metrics-client-ca\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274748    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-sys\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274792    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-root\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274795    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-textfile\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274911    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-wtmp\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.274982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.274917    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdrlb\" (UniqueName: \"kubernetes.io/projected/241d2e2a-901b-4e8f-8c85-ad365914e909-kube-api-access-hdrlb\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.275447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.275178    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-textfile\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.275447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.275304    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.275447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.275385    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241d2e2a-901b-4e8f-8c85-ad365914e909-metrics-client-ca\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.277190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.277171    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.277409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.277391    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/241d2e2a-901b-4e8f-8c85-ad365914e909-node-exporter-tls\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.329176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.329145    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdrlb\" (UniqueName: \"kubernetes.io/projected/241d2e2a-901b-4e8f-8c85-ad365914e909-kube-api-access-hdrlb\") pod \"node-exporter-mxv6r\" (UID: \"241d2e2a-901b-4e8f-8c85-ad365914e909\") " pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.419015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.418976    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mxv6r"
Nov 02 21:13:23.450131 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:23.450106    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241d2e2a_901b_4e8f_8c85_ad365914e909.slice/crio-b8a7d4ccf9d5e3f34b49f9c68f47261369afe8c5f0ca3faa3ee657cb41acbc11 WatchSource:0}: Error finding container b8a7d4ccf9d5e3f34b49f9c68f47261369afe8c5f0ca3faa3ee657cb41acbc11: Status 404 returned error can't find the container with id b8a7d4ccf9d5e3f34b49f9c68f47261369afe8c5f0ca3faa3ee657cb41acbc11
Nov 02 21:13:23.578396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.578309    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:23.578970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.578951    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5cc6793-336a-4a83-b367-65858682f31b-service-ca-bundle\") pod \"router-default-79fb5bb755-mzvm9\" (UID: \"a5cc6793-336a-4a83-b367-65858682f31b\") " pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:23.754248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.754208    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:23.795014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:23.794984    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxv6r" event={"ID":"241d2e2a-901b-4e8f-8c85-ad365914e909","Type":"ContainerStarted","Data":"b8a7d4ccf9d5e3f34b49f9c68f47261369afe8c5f0ca3faa3ee657cb41acbc11"}
Nov 02 21:13:24.243895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:24.243867    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79fb5bb755-mzvm9"]
Nov 02 21:13:24.246307 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:13:24.246273    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cc6793_336a_4a83_b367_65858682f31b.slice/crio-3e610ba286aedab8b6f7603202c2636a11b2787557a22a1fc74fa79f7d913d38 WatchSource:0}: Error finding container 3e610ba286aedab8b6f7603202c2636a11b2787557a22a1fc74fa79f7d913d38: Status 404 returned error can't find the container with id 3e610ba286aedab8b6f7603202c2636a11b2787557a22a1fc74fa79f7d913d38
Nov 02 21:13:24.801784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:24.801743    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79fb5bb755-mzvm9" event={"ID":"a5cc6793-336a-4a83-b367-65858682f31b","Type":"ContainerStarted","Data":"a124a952ffca0977e00c807038cf773e607ba9453605868aae67397ff3cc8bb1"}
Nov 02 21:13:24.801994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:24.801798    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79fb5bb755-mzvm9" event={"ID":"a5cc6793-336a-4a83-b367-65858682f31b","Type":"ContainerStarted","Data":"3e610ba286aedab8b6f7603202c2636a11b2787557a22a1fc74fa79f7d913d38"}
Nov 02 21:13:25.041524 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.041498    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pg9d6"
Nov 02 21:13:25.755190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.755145    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:25.757652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.757626    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:25.806655 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.806604    4888 generic.go:356] "Generic (PLEG): container finished" podID="241d2e2a-901b-4e8f-8c85-ad365914e909" containerID="98fc3fea52dad76f0d6d5952aab6009ab53c4a608437e8b8c8add9774405e954" exitCode=0
Nov 02 21:13:25.806840 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.806738    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxv6r" event={"ID":"241d2e2a-901b-4e8f-8c85-ad365914e909","Type":"ContainerDied","Data":"98fc3fea52dad76f0d6d5952aab6009ab53c4a608437e8b8c8add9774405e954"}
Nov 02 21:13:25.806953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.806939    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:25.808224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:25.808204    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79fb5bb755-mzvm9"
Nov 02 21:13:26.812721 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:26.812684    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxv6r" event={"ID":"241d2e2a-901b-4e8f-8c85-ad365914e909","Type":"ContainerStarted","Data":"659c9495b6670f1815cd45626f7ea9e266f0636f2c0b7112c8aa1bbdb710f4e8"}
Nov 02 21:13:26.813214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:26.812747    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxv6r" event={"ID":"241d2e2a-901b-4e8f-8c85-ad365914e909","Type":"ContainerStarted","Data":"cf0140c527cdb37e46f5c1e2980dfbb7621721b2227c1c92ffc9090609388eb2"}
Nov 02 21:13:26.922974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:26.922918    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mxv6r" podStartSLOduration=3.309928415 podStartE2EDuration="4.922904498s" totalImagesPullingTime="1.546476847s" totalInitContainerRuntime="66.499236ms" isStatefulPod=true podCreationTimestamp="2026-11-02 21:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:13:26.922024648 +0000 UTC m=+36.465816555" watchObservedRunningTime="2026-11-02 21:13:26.922904498 +0000 UTC m=+36.466696434"
Nov 02 21:13:28.593188 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.593155    4888 scope.go:122] "RemoveContainer" containerID="2ee2b1ea0a6e4eaa9a0ae9476556bc9b8cb41f07261d2934668631b755e0a15f"
Nov 02 21:13:28.823674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.823648    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:13:28.823823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.823706    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" event={"ID":"516a9d1c-5f96-4afb-9b1c-d9cf56e56703","Type":"ContainerStarted","Data":"314ed33b78b6625e56aeb5edc25be5d97fa47bcb3ff8ccbcec9aff02ba27c2d2"}
Nov 02 21:13:28.824048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.824018    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:13:28.825103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.825080    4888 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-79wcm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.21:8443/readyz\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body=
Nov 02 21:13:28.825203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:28.825127    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm" podUID="516a9d1c-5f96-4afb-9b1c-d9cf56e56703" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.21:8443/readyz\": dial tcp 10.132.0.21:8443: connect: connection refused"
Nov 02 21:13:29.740952 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:29.740878    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:29.832883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:29.832839    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-79wcm"
Nov 02 21:13:30.750895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:30.750868    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:32.836190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:32.836153    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:13:35.768978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:35.768912    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds" podUID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" containerName="registry" containerID="cri-o://f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f" gracePeriod=30
Nov 02 21:13:36.007049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.007018    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:36.060007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.059926    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.059981    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060036    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060070    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060105    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060135    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060197    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060232    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") pod \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\" (UID: \"3c91bfcd-ce7c-4910-a76f-6d018ac11daa\") "
Nov 02 21:13:36.060550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060502    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:13:36.060608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.060569    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:13:36.062843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.062794    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:13:36.062843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.062828    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:36.063038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.063015    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:36.063089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.063021    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:13:36.063089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.063058    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "kube-api-access-phvpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:36.067142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.067081    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted" pod "3c91bfcd-ce7c-4910-a76f-6d018ac11daa" (UID: "3c91bfcd-ce7c-4910-a76f-6d018ac11daa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:13:36.160993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.160959    4888 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-installation-pull-secrets\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.160993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.160988    4888 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-ca-trust-extracted\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.160993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.160998    4888 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.161217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.161009    4888 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-trusted-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.161217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.161018    4888 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-bound-sa-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.161217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.161027    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phvpc\" (UniqueName: \"kubernetes.io/projected/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-kube-api-access-phvpc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.161217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.161037    4888 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-image-registry-private-configuration\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.161217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.161049    4888 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c91bfcd-ce7c-4910-a76f-6d018ac11daa-registry-certificates\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:36.860992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.860900    4888 generic.go:356] "Generic (PLEG): container finished" podID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" containerID="f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f" exitCode=0
Nov 02 21:13:36.860992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.860960    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds"
Nov 02 21:13:36.860992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.860981    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds" event={"ID":"3c91bfcd-ce7c-4910-a76f-6d018ac11daa","Type":"ContainerDied","Data":"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"}
Nov 02 21:13:36.861540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.861060    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d4c9b5d-89lds" event={"ID":"3c91bfcd-ce7c-4910-a76f-6d018ac11daa","Type":"ContainerDied","Data":"1a1b808bef67d03cdded99a49b4787cb6cb5c30fe5af53eb45239a3e455dd29a"}
Nov 02 21:13:36.861540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.861088    4888 scope.go:122] "RemoveContainer" containerID="f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"
Nov 02 21:13:36.869165 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.869150    4888 scope.go:122] "RemoveContainer" containerID="f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"
Nov 02 21:13:36.869462 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:36.869440    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f\": container with ID starting with f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f not found: ID does not exist" containerID="f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"
Nov 02 21:13:36.869530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:36.869476    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f"} err="failed to get container status \"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f\": rpc error: code = NotFound desc = could not find container \"f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f\": container with ID starting with f1235898afbc011a07bbe5608e8edec3b23b3bc6beb3827f479642a982a3a73f not found: ID does not exist"
Nov 02 21:13:37.063492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:37.063458    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:13:37.110632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:37.110599    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86d4c9b5d-89lds"]
Nov 02 21:13:38.597157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:38.597126    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" path="/var/lib/kubelet/pods/3c91bfcd-ce7c-4910-a76f-6d018ac11daa/volumes"
Nov 02 21:13:52.258298 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:52.258261    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:13:57.856351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:57.856289    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p" podUID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" containerName="registry" containerID="cri-o://210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4" gracePeriod=30
Nov 02 21:13:58.114330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.114266    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:58.228440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228401    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228449    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228530    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228559    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228582    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228609    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.228675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.228627    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") pod \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\" (UID: \"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83\") "
Nov 02 21:13:58.229199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.229148    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:13:58.229547 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.229513    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:13:58.231098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.231071    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:13:58.231364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.231343    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:13:58.231463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.231403    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "kube-api-access-n8gxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:58.231535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.231460    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:58.231535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.231478    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:13:58.236038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.236013    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted" pod "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" (UID: "f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:13:58.329153 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329121    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8gxn\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-kube-api-access-n8gxn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329153 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329151    4888 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-certificates\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329153 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329161    4888 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-installation-pull-secrets\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329171    4888 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-registry-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329181    4888 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-ca-trust-extracted\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329191    4888 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-image-registry-private-configuration\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329200    4888 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-bound-sa-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.329367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.329209    4888 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83-trusted-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:13:58.947943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.947913    4888 generic.go:356] "Generic (PLEG): container finished" podID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" containerID="210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4" exitCode=0
Nov 02 21:13:58.948359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.947958    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p"
Nov 02 21:13:58.948359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.948002    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p" event={"ID":"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83","Type":"ContainerDied","Data":"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"}
Nov 02 21:13:58.948359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.948058    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b547d4cf-j2p2p" event={"ID":"f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83","Type":"ContainerDied","Data":"e82a9b6c2b0cec679bc4da12e8ba0f1e789784acff7624733cd646bfb71b8475"}
Nov 02 21:13:58.948359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.948086    4888 scope.go:122] "RemoveContainer" containerID="210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"
Nov 02 21:13:58.956106 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.956085    4888 scope.go:122] "RemoveContainer" containerID="210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"
Nov 02 21:13:58.956370 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:13:58.956351    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4\": container with ID starting with 210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4 not found: ID does not exist" containerID="210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"
Nov 02 21:13:58.956419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:58.956380    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4"} err="failed to get container status \"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4\": rpc error: code = NotFound desc = could not find container \"210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4\": container with ID starting with 210089ec4f55ab3b7329df16cd8609c28be0d19c0f8fee388f6dbce1c0eba0c4 not found: ID does not exist"
Nov 02 21:13:59.150110 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:59.150069    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:13:59.197473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:13:59.197435    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64b547d4cf-j2p2p"]
Nov 02 21:14:00.597720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:00.597688    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" path="/var/lib/kubelet/pods/f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83/volumes"
Nov 02 21:14:06.977248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:06.977213    4888 generic.go:356] "Generic (PLEG): container finished" podID="7b91894b-7a38-4d8f-be77-d76177da71bb" containerID="5590c0f49257a88c03b580f419359ad9f8eedf152db668e21175c03f95d5da5e" exitCode=0
Nov 02 21:14:06.977633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:06.977275    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9" event={"ID":"7b91894b-7a38-4d8f-be77-d76177da71bb","Type":"ContainerDied","Data":"5590c0f49257a88c03b580f419359ad9f8eedf152db668e21175c03f95d5da5e"}
Nov 02 21:14:06.977633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:06.977590    4888 scope.go:122] "RemoveContainer" containerID="5590c0f49257a88c03b580f419359ad9f8eedf152db668e21175c03f95d5da5e"
Nov 02 21:14:07.983918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:07.983881    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vrsr9" event={"ID":"7b91894b-7a38-4d8f-be77-d76177da71bb","Type":"ContainerStarted","Data":"e4770e79c1e344ce982d0104a907ed60bc44d785f0ccf9cad5f1f38a48918a89"}
Nov 02 21:14:22.036691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:22.036601    4888 generic.go:356] "Generic (PLEG): container finished" podID="921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd" containerID="e7a29e01412a7b176f301639f1de8afe8d33fc87b27eeece1ea12b6707012885" exitCode=0
Nov 02 21:14:22.037193 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:22.036679    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-42685" event={"ID":"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd","Type":"ContainerDied","Data":"e7a29e01412a7b176f301639f1de8afe8d33fc87b27eeece1ea12b6707012885"}
Nov 02 21:14:22.037193 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:22.037122    4888 scope.go:122] "RemoveContainer" containerID="e7a29e01412a7b176f301639f1de8afe8d33fc87b27eeece1ea12b6707012885"
Nov 02 21:14:22.259254 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:14:22.259219    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:14:23.041914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.041880    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-42685" event={"ID":"921fd5a0-8fdb-4da9-a7bd-8debf8c5bdcd","Type":"ContainerStarted","Data":"3305251d037fb247948a1d1f978b420bda603963c7eba61c8defa897a6bcadc0"}
Nov 02 21:14:23.043174 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.043153    4888 generic.go:356] "Generic (PLEG): container finished" podID="cbea11f1-0e1c-4cdd-be0e-c33538c05fee" containerID="00d9b3fc4b038bd537d3471559eb0c780be37fa3c6c24c162eea5de9682c94ca" exitCode=0
Nov 02 21:14:23.043276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.043229    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw" event={"ID":"cbea11f1-0e1c-4cdd-be0e-c33538c05fee","Type":"ContainerDied","Data":"00d9b3fc4b038bd537d3471559eb0c780be37fa3c6c24c162eea5de9682c94ca"}
Nov 02 21:14:23.043507 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.043494    4888 scope.go:122] "RemoveContainer" containerID="00d9b3fc4b038bd537d3471559eb0c780be37fa3c6c24c162eea5de9682c94ca"
Nov 02 21:14:23.194449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194419    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-wzckv-xxrt2"]
Nov 02 21:14:23.194771 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194755    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" containerName="registry"
Nov 02 21:14:23.195024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194774    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" containerName="registry"
Nov 02 21:14:23.195024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194788    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" containerName="registry"
Nov 02 21:14:23.195024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194796    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" containerName="registry"
Nov 02 21:14:23.195024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194908    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f36b69cc-b9fa-410b-a0f8-7dc6cc2e3c83" containerName="registry"
Nov 02 21:14:23.195024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.194921    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c91bfcd-ce7c-4910-a76f-6d018ac11daa" containerName="registry"
Nov 02 21:14:23.198175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.198150    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.250323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.250290    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-wzckv-xxrt2"]
Nov 02 21:14:23.315348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.315266    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.315348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.315315    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.315514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.315395    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.315514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.315464    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djc5t\" (UniqueName: \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.416230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.416184    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.416389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.416244    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djc5t\" (UniqueName: \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.416389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.416295    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.416389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.416350    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.416803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.416783    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.417049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.417029    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.417122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.417091    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.469981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.469952    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djc5t\" (UniqueName: \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\") pod \"periodic-gathering-wzckv-xxrt2\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") " pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.508837 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.508800    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:14:23.867631 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:23.867605    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-wzckv-xxrt2"]
Nov 02 21:14:23.870005 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:14:23.869974    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc77f8f2_60d3_4ee2_a07b_57e06116a787.slice/crio-1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df WatchSource:0}: Error finding container 1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df: Status 404 returned error can't find the container with id 1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df
Nov 02 21:14:24.051186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:24.051154    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-bccqw" event={"ID":"cbea11f1-0e1c-4cdd-be0e-c33538c05fee","Type":"ContainerStarted","Data":"1d997c209d117451812738a5ee49547cc2ce21ca6678065b022eedbc0f39ba67"}
Nov 02 21:14:24.052754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:24.052722    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wzckv-xxrt2" event={"ID":"bc77f8f2-60d3-4ee2-a07b-57e06116a787","Type":"ContainerStarted","Data":"8cd11960ff955d1c95c8fae9e2f8b98e5e1a142272dc4fe2a1576ec62de816ab"}
Nov 02 21:14:24.052909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:24.052777    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wzckv-xxrt2" event={"ID":"bc77f8f2-60d3-4ee2-a07b-57e06116a787","Type":"ContainerStarted","Data":"1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df"}
Nov 02 21:14:24.268174 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:24.268119    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-wzckv-xxrt2" podStartSLOduration=1.26810511 podStartE2EDuration="1.26810511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:14:24.267632786 +0000 UTC m=+93.811424702" watchObservedRunningTime="2026-11-02 21:14:24.26810511 +0000 UTC m=+93.811897020"
Nov 02 21:14:25.337126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:25.337098    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79fb5bb755-mzvm9_a5cc6793-336a-4a83-b367-65858682f31b/router/0.log"
Nov 02 21:14:25.599397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:25.599319    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nh249_3cc1883d-1d29-4185-9ea6-fda6b9e021d1/serve-healthcheck-canary/0.log"
Nov 02 21:14:34.271127 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.271084    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-68946565d6-kpkp7"]
Nov 02 21:14:34.274820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.274797    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.318646 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.318608    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68946565d6-kpkp7"]
Nov 02 21:14:34.328870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.328824    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 02 21:14:34.329026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.328910    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 02 21:14:34.329026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.328916    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 02 21:14:34.329026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.328942    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 02 21:14:34.329196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.329182    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8t48c\""
Nov 02 21:14:34.329259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.329250    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 02 21:14:34.406571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406534    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nggc9\" (UniqueName: \"kubernetes.io/projected/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-kube-api-access-nggc9\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406573    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406595    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-metrics-client-ca\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406610    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406749    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-serving-certs-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406828    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406887    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.406978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.406903    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-federate-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.420863 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.420811    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 02 21:14:34.508013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.507973    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-federate-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508022    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nggc9\" (UniqueName: \"kubernetes.io/projected/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-kube-api-access-nggc9\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508041    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508071    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-metrics-client-ca\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508096    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508130    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-serving-certs-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508180    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508224    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.508907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.508876    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-metrics-client-ca\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.509259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.509214    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-serving-certs-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.509259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.509222    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.510902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.510876    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-federate-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.511016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.510995    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-telemeter-client-tls\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.511060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.511048    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.511094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.511059    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.562475 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.562405    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nggc9\" (UniqueName: \"kubernetes.io/projected/c4d67dc9-550f-48a3-bfb6-7008fe02c7e9-kube-api-access-nggc9\") pod \"telemeter-client-68946565d6-kpkp7\" (UID: \"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9\") " pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:34.592298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:34.592262    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7"
Nov 02 21:14:35.049614 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:35.049579    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68946565d6-kpkp7"]
Nov 02 21:14:35.097754 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:14:35.097721    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d67dc9_550f_48a3_bfb6_7008fe02c7e9.slice/crio-4ef63e60459e5ab0b41692a50e22742e34c0e36815715b7bdb17afd49d94108b WatchSource:0}: Error finding container 4ef63e60459e5ab0b41692a50e22742e34c0e36815715b7bdb17afd49d94108b: Status 404 returned error can't find the container with id 4ef63e60459e5ab0b41692a50e22742e34c0e36815715b7bdb17afd49d94108b
Nov 02 21:14:36.099259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:36.099223    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7" event={"ID":"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9","Type":"ContainerStarted","Data":"4ef63e60459e5ab0b41692a50e22742e34c0e36815715b7bdb17afd49d94108b"}
Nov 02 21:14:39.110522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:39.110481    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7" event={"ID":"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9","Type":"ContainerStarted","Data":"93177f2b73f75c72b2e5fb93c3c79a6d1649c4c12a4084ab5edaf0d8025a56c0"}
Nov 02 21:14:41.122102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:41.122071    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7" event={"ID":"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9","Type":"ContainerStarted","Data":"31f32bda849b0f384442ed468a9ef6a58e146db5f8448a65d47f9d57fe975680"}
Nov 02 21:14:42.128047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:42.128005    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7" event={"ID":"c4d67dc9-550f-48a3-bfb6-7008fe02c7e9","Type":"ContainerStarted","Data":"160708aff4160b35d771d5adc6d53aaad9e17e858ac87e6a685c6603bd4bd21c"}
Nov 02 21:14:42.335506 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:14:42.335442    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-68946565d6-kpkp7" podStartSLOduration=2.493180245 podStartE2EDuration="8.335429832s" totalImagesPullingTime="5.842249587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:14:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:14:42.287584446 +0000 UTC m=+111.831376362" watchObservedRunningTime="2026-11-02 21:14:42.335429832 +0000 UTC m=+111.879221744"
Nov 02 21:14:52.258995 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:14:52.258955    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:15:22.259116 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:15:22.259032    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:15:51.387788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:51.387750    4888 generic.go:356] "Generic (PLEG): container finished" podID="bc77f8f2-60d3-4ee2-a07b-57e06116a787" containerID="8cd11960ff955d1c95c8fae9e2f8b98e5e1a142272dc4fe2a1576ec62de816ab" exitCode=0
Nov 02 21:15:51.388213 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:51.387823    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wzckv-xxrt2" event={"ID":"bc77f8f2-60d3-4ee2-a07b-57e06116a787","Type":"ContainerDied","Data":"8cd11960ff955d1c95c8fae9e2f8b98e5e1a142272dc4fe2a1576ec62de816ab"}
Nov 02 21:15:52.258832 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:15:52.258799    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:15:52.527289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.527263    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:15:52.609338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609305    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\" (UniqueName: \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\") pod \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") "
Nov 02 21:15:52.609535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609350    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\") pod \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") "
Nov 02 21:15:52.609535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609369    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\") pod \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") "
Nov 02 21:15:52.609535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609492    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\") pod \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\" (UID: \"bc77f8f2-60d3-4ee2-a07b-57e06116a787\") "
Nov 02 21:15:52.609799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609768    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path" pod "bc77f8f2-60d3-4ee2-a07b-57e06116a787" (UID: "bc77f8f2-60d3-4ee2-a07b-57e06116a787"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:15:52.609799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609782    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle" pod "bc77f8f2-60d3-4ee2-a07b-57e06116a787" (UID: "bc77f8f2-60d3-4ee2-a07b-57e06116a787"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:15:52.609974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.609774    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle" pod "bc77f8f2-60d3-4ee2-a07b-57e06116a787" (UID: "bc77f8f2-60d3-4ee2-a07b-57e06116a787"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:15:52.611611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.611591    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t" pod "bc77f8f2-60d3-4ee2-a07b-57e06116a787" (UID: "bc77f8f2-60d3-4ee2-a07b-57e06116a787"). InnerVolumeSpecName "kube-api-access-djc5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:15:52.710552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.710516    4888 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bc77f8f2-60d3-4ee2-a07b-57e06116a787-archives-path\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:15:52.710552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.710544    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djc5t\" (UniqueName: \"kubernetes.io/projected/bc77f8f2-60d3-4ee2-a07b-57e06116a787-kube-api-access-djc5t\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:15:52.710552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.710554    4888 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-service-ca-bundle\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:15:52.710552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:52.710563    4888 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc77f8f2-60d3-4ee2-a07b-57e06116a787-trusted-ca-bundle\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:15:53.396182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:53.396144    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wzckv-xxrt2" event={"ID":"bc77f8f2-60d3-4ee2-a07b-57e06116a787","Type":"ContainerDied","Data":"1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df"}
Nov 02 21:15:53.396182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:53.396164    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wzckv-xxrt2"
Nov 02 21:15:53.396468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:15:53.396193    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f873fc8ae1ed3dc29c3710933b9e25d8344513c37982d984a03dd7bd64585df"
Nov 02 21:16:22.258746 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:16:22.258707    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:16:52.258258 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:16:52.258221    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:17:22.258908 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:17:22.258844    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:17:27.547171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.547137    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"]
Nov 02 21:17:27.547571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.547445    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bc77f8f2-60d3-4ee2-a07b-57e06116a787" containerName="insights-gathering"
Nov 02 21:17:27.547571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.547534    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc77f8f2-60d3-4ee2-a07b-57e06116a787" containerName="insights-gathering"
Nov 02 21:17:27.547661 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.547596    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc77f8f2-60d3-4ee2-a07b-57e06116a787" containerName="insights-gathering"
Nov 02 21:17:27.550727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.550706    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.650907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.650838    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-dd8t7\""
Nov 02 21:17:27.650907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.650840    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 02 21:17:27.650907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.650901    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 02 21:17:27.651817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.651792    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"]
Nov 02 21:17:27.657824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.657792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvj8\" (UniqueName: \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.657936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.657868    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.658036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.658008    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.758548 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.758518    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.758715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.758560    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.758715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.758590    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvj8\" (UniqueName: \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.758996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.758973    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.759085 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.759020    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.816008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.815938    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvj8\" (UniqueName: \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:27.866715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:27.866677    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:28.101035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.100951    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:17:28.104431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.104413    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.157021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.156985    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 21:17:28.157021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.157011    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 21:17:28.157221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.156989    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 21:17:28.157412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.157390    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 21:17:28.157491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.157476    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nwm6f\""
Nov 02 21:17:28.193736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.193699    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:17:28.229558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.229524    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"]
Nov 02 21:17:28.241488 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:17:28.241460    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a202f8_ef26_42d2_9656_366af6f59ea8.slice/crio-17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9 WatchSource:0}: Error finding container 17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9: Status 404 returned error can't find the container with id 17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9
Nov 02 21:17:28.263091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263053    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263091    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263208    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263235    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263267    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhxs\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263386    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.263448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.263425    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.363875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.363777    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.363875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.363820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.363961    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.363999    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.364071    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.364101    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.364142    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhxs\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.364525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.364499    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.366258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.366235    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.366467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.366444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.366467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.366461    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.366581 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.366505    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.418421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.418386    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.418618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.418597    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhxs\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\") pod \"istiod-openshift-gateway-7876bfcb74-466l7\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.716387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.716353    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:28.741901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:28.741867    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv" event={"ID":"d4a202f8-ef26-42d2-9656-366af6f59ea8","Type":"ContainerStarted","Data":"17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9"}
Nov 02 21:17:29.129070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:29.129043    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:17:29.130822 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:17:29.130796    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4245ee27_1581_4c84_8f94_bc35b4f30622.slice/crio-3c0e6db97b7f6e4b2c62b10e907b21ad135ba7fa4747c2a5990261891578a086 WatchSource:0}: Error finding container 3c0e6db97b7f6e4b2c62b10e907b21ad135ba7fa4747c2a5990261891578a086: Status 404 returned error can't find the container with id 3c0e6db97b7f6e4b2c62b10e907b21ad135ba7fa4747c2a5990261891578a086
Nov 02 21:17:29.747975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:29.747924    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" event={"ID":"4245ee27-1581-4c84-8f94-bc35b4f30622","Type":"ContainerStarted","Data":"3c0e6db97b7f6e4b2c62b10e907b21ad135ba7fa4747c2a5990261891578a086"}
Nov 02 21:17:33.439842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:33.439806    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:17:33.440188 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:33.439911    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:17:33.770397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:33.770310    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" event={"ID":"4245ee27-1581-4c84-8f94-bc35b4f30622","Type":"ContainerStarted","Data":"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"}
Nov 02 21:17:33.770560 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:33.770513    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:33.928957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:33.928892    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" podStartSLOduration=1.623168003 podStartE2EDuration="5.928875332s" totalImagesPullingTime="4.305707329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:17:33.880386638 +0000 UTC m=+283.424178553" watchObservedRunningTime="2026-11-02 21:17:33.928875332 +0000 UTC m=+283.472667248"
Nov 02 21:17:34.777628 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:34.777596    4888 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-466l7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:17:34.778125 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:34.777658    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:17:35.780815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:35.780779    4888 generic.go:356] "Generic (PLEG): container finished" podID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerID="1414824beaee92f1ba5ec296ed8b29f9869d97e30913134e4a4c3f2a2686d859" exitCode=0
Nov 02 21:17:35.781236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:35.780873    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv" event={"ID":"d4a202f8-ef26-42d2-9656-366af6f59ea8","Type":"ContainerDied","Data":"1414824beaee92f1ba5ec296ed8b29f9869d97e30913134e4a4c3f2a2686d859"}
Nov 02 21:17:37.775893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:37.775836    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:17:39.799909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:39.799877    4888 generic.go:356] "Generic (PLEG): container finished" podID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerID="0e5bfad7b7944636b0996bd2a1cf961a86b3a05b6c62a9c978e29b3073093a8a" exitCode=0
Nov 02 21:17:39.800385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:39.799953    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv" event={"ID":"d4a202f8-ef26-42d2-9656-366af6f59ea8","Type":"ContainerDied","Data":"0e5bfad7b7944636b0996bd2a1cf961a86b3a05b6c62a9c978e29b3073093a8a"}
Nov 02 21:17:46.833222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:46.833188    4888 generic.go:356] "Generic (PLEG): container finished" podID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerID="2cdbcde0ecadeb265cec38a28987c0ce309a7e0a3c4577184df492cda502c69e" exitCode=0
Nov 02 21:17:46.833601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:46.833264    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv" event={"ID":"d4a202f8-ef26-42d2-9656-366af6f59ea8","Type":"ContainerDied","Data":"2cdbcde0ecadeb265cec38a28987c0ce309a7e0a3c4577184df492cda502c69e"}
Nov 02 21:17:47.965322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:47.965294    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:48.032943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.032911    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\") pod \"d4a202f8-ef26-42d2-9656-366af6f59ea8\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") "
Nov 02 21:17:48.033107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.032960    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\") pod \"d4a202f8-ef26-42d2-9656-366af6f59ea8\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") "
Nov 02 21:17:48.033107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.033004    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\" (UniqueName: \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\") pod \"d4a202f8-ef26-42d2-9656-366af6f59ea8\" (UID: \"d4a202f8-ef26-42d2-9656-366af6f59ea8\") "
Nov 02 21:17:48.033387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.033363    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle" pod "d4a202f8-ef26-42d2-9656-366af6f59ea8" (UID: "d4a202f8-ef26-42d2-9656-366af6f59ea8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:17:48.035253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.035228    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8" pod "d4a202f8-ef26-42d2-9656-366af6f59ea8" (UID: "d4a202f8-ef26-42d2-9656-366af6f59ea8"). InnerVolumeSpecName "kube-api-access-jcvj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:17:48.038200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.038174    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util" pod "d4a202f8-ef26-42d2-9656-366af6f59ea8" (UID: "d4a202f8-ef26-42d2-9656-366af6f59ea8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:17:48.133569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.133478    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-util\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:17:48.133569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.133511    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a202f8-ef26-42d2-9656-366af6f59ea8-bundle\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:17:48.133569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.133521    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcvj8\" (UniqueName: \"kubernetes.io/projected/d4a202f8-ef26-42d2-9656-366af6f59ea8-kube-api-access-jcvj8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:17:48.843510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.843473    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv" event={"ID":"d4a202f8-ef26-42d2-9656-366af6f59ea8","Type":"ContainerDied","Data":"17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9"}
Nov 02 21:17:48.843510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.843500    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444z5jv"
Nov 02 21:17:48.843714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:48.843530    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17ea070c9833bf4acc6901635fb54372ccbc92e76ebbda57ab1231ad36f044c9"
Nov 02 21:17:50.617892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:50.617864    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:17:50.618354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:17:50.617931    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:17:52.258512 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:17:52.258472    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:18:22.259261 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:18:22.259176    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:18:32.988549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988519    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"]
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988859    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="pull"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988873    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="pull"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988905    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="util"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988910    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="util"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988917    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="extract"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988921    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="extract"
Nov 02 21:18:32.989047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.988974    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4a202f8-ef26-42d2-9656-366af6f59ea8" containerName="extract"
Nov 02 21:18:32.991944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:32.991927    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.036656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.036623    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"]
Nov 02 21:18:33.041564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.041535    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 21:18:33.041678 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.041541    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 21:18:33.079398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.079338    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 21:18:33.079572 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.079538    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 21:18:33.079645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.079582    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 21:18:33.079645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.079620    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-mt8kr\""
Nov 02 21:18:33.107157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107118    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.107312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107174    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-manifests\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.107312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107292    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-tmp\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.107387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmq6h\" (UniqueName: \"kubernetes.io/projected/6e4e86d9-3c73-4e62-8005-b861ef3d8460-kube-api-access-fmq6h\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.107421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107385    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-metrics-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.107453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.107418    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-webhook-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208273 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208227    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-tmp\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208283    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmq6h\" (UniqueName: \"kubernetes.io/projected/6e4e86d9-3c73-4e62-8005-b861ef3d8460-kube-api-access-fmq6h\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208309    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-metrics-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208334    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-webhook-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208374    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208429    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-manifests\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.208826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.208793    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-manifests\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.210919 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.210896    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-metrics-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.211025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.210934    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-webhook-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.211066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.211019    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4e86d9-3c73-4e62-8005-b861ef3d8460-tmp\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.211267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.211250    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e4e86d9-3c73-4e62-8005-b861ef3d8460-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.263859 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.263773    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmq6h\" (UniqueName: \"kubernetes.io/projected/6e4e86d9-3c73-4e62-8005-b861ef3d8460-kube-api-access-fmq6h\") pod \"opendatahub-operator-controller-manager-676bcb69b5-l2jv7\" (UID: \"6e4e86d9-3c73-4e62-8005-b861ef3d8460\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.306613 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.306576    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:33.779371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.779344    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"]
Nov 02 21:18:33.781210 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:18:33.781182    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4e86d9_3c73_4e62_8005_b861ef3d8460.slice/crio-33a6f86fbd0565be0849116f0a93b88d451b525dd59419696cfaf6ca3f447cbb WatchSource:0}: Error finding container 33a6f86fbd0565be0849116f0a93b88d451b525dd59419696cfaf6ca3f447cbb: Status 404 returned error can't find the container with id 33a6f86fbd0565be0849116f0a93b88d451b525dd59419696cfaf6ca3f447cbb
Nov 02 21:18:33.782996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:33.782977    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:18:34.025516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:34.025478    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7" event={"ID":"6e4e86d9-3c73-4e62-8005-b861ef3d8460","Type":"ContainerStarted","Data":"33a6f86fbd0565be0849116f0a93b88d451b525dd59419696cfaf6ca3f447cbb"}
Nov 02 21:18:38.047886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:38.047786    4888 generic.go:356] "Generic (PLEG): container finished" podID="6e4e86d9-3c73-4e62-8005-b861ef3d8460" containerID="8eaefedf1f46a3878c48d83d6f0baaa106b6051010a7e2ddde04f61e64c7d594" exitCode=0
Nov 02 21:18:38.047886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:38.047876    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7" event={"ID":"6e4e86d9-3c73-4e62-8005-b861ef3d8460","Type":"ContainerDied","Data":"8eaefedf1f46a3878c48d83d6f0baaa106b6051010a7e2ddde04f61e64c7d594"}
Nov 02 21:18:39.053926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:39.053891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7" event={"ID":"6e4e86d9-3c73-4e62-8005-b861ef3d8460","Type":"ContainerStarted","Data":"efcdbacfdf23b2f8b0ee10f67504f4c9ea5c6b89d7c2afecef9c0b5d646aaeff"}
Nov 02 21:18:39.054359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:39.054049    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:39.274008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:39.273932    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7" podStartSLOduration=2.552314782 podStartE2EDuration="7.273917326s" totalImagesPullingTime="4.310551515s" totalInitContainerRuntime="411.051029ms" isStatefulPod=false podCreationTimestamp="2026-11-02 21:18:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:18:39.225945548 +0000 UTC m=+348.769737464" watchObservedRunningTime="2026-11-02 21:18:39.273917326 +0000 UTC m=+348.817709242"
Nov 02 21:18:50.060180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:18:50.060149    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-676bcb69b5-l2jv7"
Nov 02 21:18:52.258752 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:18:52.258709    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:19:02.339932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.339891    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"]
Nov 02 21:19:02.343605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.343578    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.396894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.396863    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"]
Nov 02 21:19:02.443067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443029    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 02 21:19:02.443253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443070    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 02 21:19:02.443253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443146    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f9p44\""
Nov 02 21:19:02.443253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443234    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tmp\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.443393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443339    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj96t\" (UniqueName: \"kubernetes.io/projected/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-kube-api-access-wj96t\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.443393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.443386    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tls-certs\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.543909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.543870    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj96t\" (UniqueName: \"kubernetes.io/projected/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-kube-api-access-wj96t\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.544109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.543926    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tls-certs\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.544109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.543955    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tmp\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.546170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.546147    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tmp\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.546432 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.546414    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-tls-certs\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.599175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.599100    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj96t\" (UniqueName: \"kubernetes.io/projected/b342fdb0-cb2a-4481-8d94-f2e6131ed2e9-kube-api-access-wj96t\") pod \"kube-auth-proxy-795b66c7fb-8b5vf\" (UID: \"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9\") " pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:02.655842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:02.655807    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"
Nov 02 21:19:03.024513 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:03.024415    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf"]
Nov 02 21:19:03.027157 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:19:03.027132    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb342fdb0_cb2a_4481_8d94_f2e6131ed2e9.slice/crio-ff4b5f9a2f58da47b2756bd2251f8bdae3feb62e2cf3fb66fe1940554d140800 WatchSource:0}: Error finding container ff4b5f9a2f58da47b2756bd2251f8bdae3feb62e2cf3fb66fe1940554d140800: Status 404 returned error can't find the container with id ff4b5f9a2f58da47b2756bd2251f8bdae3feb62e2cf3fb66fe1940554d140800
Nov 02 21:19:03.153146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:03.153113    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf" event={"ID":"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9","Type":"ContainerStarted","Data":"ff4b5f9a2f58da47b2756bd2251f8bdae3feb62e2cf3fb66fe1940554d140800"}
Nov 02 21:19:06.171713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:06.171671    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf" event={"ID":"b342fdb0-cb2a-4481-8d94-f2e6131ed2e9","Type":"ContainerStarted","Data":"5c84135114de3869a3587c91fbc661580b7d1b5709983b54f0ce25b83ca7a351"}
Nov 02 21:19:06.282544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:06.282470    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf" podStartSLOduration=1.456204949 podStartE2EDuration="4.282455166s" totalImagesPullingTime="2.826250217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:19:06.28152887 +0000 UTC m=+375.825320785" watchObservedRunningTime="2026-11-02 21:19:06.282455166 +0000 UTC m=+375.826247081"
Nov 02 21:19:13.599748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.599713    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"]
Nov 02 21:19:13.606498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.606476    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.634455 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.634418    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.634650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.634464    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.634650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.634596    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c744b7f7-7c53-453f-b8cf-d99b04c61077-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.634650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.634642    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68llr\" (UniqueName: \"kubernetes.io/projected/c744b7f7-7c53-453f-b8cf-d99b04c61077-kube-api-access-68llr\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.704393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.704344    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 21:19:13.704393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.704368    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 21:19:13.704629 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.704454    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-5h2ph\""
Nov 02 21:19:13.705691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.705671    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"]
Nov 02 21:19:13.735600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.735564    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c744b7f7-7c53-453f-b8cf-d99b04c61077-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.735600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.735604    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68llr\" (UniqueName: \"kubernetes.io/projected/c744b7f7-7c53-453f-b8cf-d99b04c61077-kube-api-access-68llr\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.735822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.735648    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.735822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.735672    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.736074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.736058    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.736117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.736100    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c744b7f7-7c53-453f-b8cf-d99b04c61077-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.799963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.799916    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 02 21:19:13.802531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.802502    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68llr\" (UniqueName: \"kubernetes.io/projected/c744b7f7-7c53-453f-b8cf-d99b04c61077-kube-api-access-68llr\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.806839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.806811    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c744b7f7-7c53-453f-b8cf-d99b04c61077-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q6cgf\" (UID: \"c744b7f7-7c53-453f-b8cf-d99b04c61077\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:13.918476 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:13.918448    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:14.339126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:14.339099    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"]
Nov 02 21:19:14.340781 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:19:14.340754    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc744b7f7_7c53_453f_b8cf_d99b04c61077.slice/crio-c6acbfc42c7e7cd216964e58586908604853595bcd24593da4df8a818bddf9c1 WatchSource:0}: Error finding container c6acbfc42c7e7cd216964e58586908604853595bcd24593da4df8a818bddf9c1: Status 404 returned error can't find the container with id c6acbfc42c7e7cd216964e58586908604853595bcd24593da4df8a818bddf9c1
Nov 02 21:19:15.211104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:15.211065    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf" event={"ID":"c744b7f7-7c53-453f-b8cf-d99b04c61077","Type":"ContainerStarted","Data":"c6acbfc42c7e7cd216964e58586908604853595bcd24593da4df8a818bddf9c1"}
Nov 02 21:19:17.220960 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:17.220920    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf" event={"ID":"c744b7f7-7c53-453f-b8cf-d99b04c61077","Type":"ContainerStarted","Data":"3fdaa9310a2495790ff395ade1f1cc6ce109467b6fe8e87aad9d149e4279c855"}
Nov 02 21:19:17.221396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:17.221010    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:17.377628 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:17.377522    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf" podStartSLOduration=1.668627613 podStartE2EDuration="4.377504102s" totalImagesPullingTime="2.708876489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:19:17.376933334 +0000 UTC m=+386.920725250" watchObservedRunningTime="2026-11-02 21:19:17.377504102 +0000 UTC m=+386.921296017"
Nov 02 21:19:22.259108 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:19:22.259069    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:19:38.227003 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:38.226918    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q6cgf"
Nov 02 21:19:49.434315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.434275    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:19:49.441904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.441868    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.491925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.491883    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 21:19:49.491925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.491915    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z6pqx\""
Nov 02 21:19:49.492139 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.492048    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 21:19:49.539944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.539905    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 21:19:49.541473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.541441    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:19:49.545398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.545374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.545492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.545471    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svgf9\" (UniqueName: \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.545576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.545552    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.646523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.646483    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svgf9\" (UniqueName: \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.646713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.646559    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.646713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.646588    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.649195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.649172    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.649349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.649326    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.701387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.701306    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svgf9\" (UniqueName: \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\") pod \"maas-controller-579cf7c6d-nkk94\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") " pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.755028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.754987    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:49.818760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.818710    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 21:19:49.823328 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.823299    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:49.875286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.875255    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 21:19:49.949601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.949564    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:49.949601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.949606    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ggnd\" (UniqueName: \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:49.949875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:49.949634    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.050914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.050786    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.051075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.050923    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.051075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.050956    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ggnd\" (UniqueName: \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.053394 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.053368    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.053503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.053412    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.105204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.105174    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ggnd\" (UniqueName: \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\") pod \"maas-controller-9f8dffcbc-cnnfl\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") " pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.116708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.116680    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:19:50.118617 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:19:50.118577    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2667584_632e_4394_80a2_175d8986ac4d.slice/crio-da8b9f1cf93f99497036d4a24dfa46bf889642d13c15949029eaf8cd90460f13 WatchSource:0}: Error finding container da8b9f1cf93f99497036d4a24dfa46bf889642d13c15949029eaf8cd90460f13: Status 404 returned error can't find the container with id da8b9f1cf93f99497036d4a24dfa46bf889642d13c15949029eaf8cd90460f13
Nov 02 21:19:50.139957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.139918    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:50.364122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.364035    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-nkk94" event={"ID":"c2667584-632e-4394-80a2-175d8986ac4d","Type":"ContainerStarted","Data":"da8b9f1cf93f99497036d4a24dfa46bf889642d13c15949029eaf8cd90460f13"}
Nov 02 21:19:50.504369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:50.504344    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 21:19:50.506555 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:19:50.506528    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7f5139_0479_4321_9bf7_49f1dd103ad2.slice/crio-0f7751b8b23d848b922416c315b3dfb24837ac62b6ea8381d63f7fa5f6574454 WatchSource:0}: Error finding container 0f7751b8b23d848b922416c315b3dfb24837ac62b6ea8381d63f7fa5f6574454: Status 404 returned error can't find the container with id 0f7751b8b23d848b922416c315b3dfb24837ac62b6ea8381d63f7fa5f6574454
Nov 02 21:19:51.369930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:51.369891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerStarted","Data":"0f7751b8b23d848b922416c315b3dfb24837ac62b6ea8381d63f7fa5f6574454"}
Nov 02 21:19:52.295070 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:19:52.294962    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:19:53.987185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:53.987149    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 21:19:54.390305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.390202    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerStarted","Data":"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"}
Nov 02 21:19:54.390477 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.390304    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:19:54.391820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.391795    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-nkk94" event={"ID":"c2667584-632e-4394-80a2-175d8986ac4d","Type":"ContainerStarted","Data":"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"}
Nov 02 21:19:54.391956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.391868    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:19:54.499762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.499709    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" podStartSLOduration=1.920291285 podStartE2EDuration="5.499696025s" totalImagesPullingTime="3.57940474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:19:54.498670062 +0000 UTC m=+424.042461979" watchObservedRunningTime="2026-11-02 21:19:54.499696025 +0000 UTC m=+424.043487940"
Nov 02 21:19:54.607760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:19:54.607708    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-nkk94" podStartSLOduration=1.7901640890000001 podStartE2EDuration="5.607693519s" totalImagesPullingTime="3.81752943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:19:54.606584018 +0000 UTC m=+424.150375934" watchObservedRunningTime="2026-11-02 21:19:54.607693519 +0000 UTC m=+424.151485435"
Nov 02 21:20:05.398435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.398401    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:20:05.398946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.398487    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:20:05.558703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.558660    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:20:05.558932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.558910    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-nkk94" podUID="c2667584-632e-4394-80a2-175d8986ac4d" containerName="manager" containerID="cri-o://a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4" gracePeriod=30
Nov 02 21:20:05.791321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.791292    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:20:05.879752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.879718    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\") pod \"c2667584-632e-4394-80a2-175d8986ac4d\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") "
Nov 02 21:20:05.879983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.879803    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\") pod \"c2667584-632e-4394-80a2-175d8986ac4d\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") "
Nov 02 21:20:05.879983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.879891    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\" (UniqueName: \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\") pod \"c2667584-632e-4394-80a2-175d8986ac4d\" (UID: \"c2667584-632e-4394-80a2-175d8986ac4d\") "
Nov 02 21:20:05.882028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.881998    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert" pod "c2667584-632e-4394-80a2-175d8986ac4d" (UID: "c2667584-632e-4394-80a2-175d8986ac4d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:20:05.882028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.882017    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9" pod "c2667584-632e-4394-80a2-175d8986ac4d" (UID: "c2667584-632e-4394-80a2-175d8986ac4d"). InnerVolumeSpecName "kube-api-access-svgf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:20:05.882171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.882091    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs" pod "c2667584-632e-4394-80a2-175d8986ac4d" (UID: "c2667584-632e-4394-80a2-175d8986ac4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:20:05.980778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.980686    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svgf9\" (UniqueName: \"kubernetes.io/projected/c2667584-632e-4394-80a2-175d8986ac4d-kube-api-access-svgf9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:05.980778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.980722    4888 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-webhook-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:05.980778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:05.980730    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2667584-632e-4394-80a2-175d8986ac4d-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:06.445198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.445161    4888 generic.go:356] "Generic (PLEG): container finished" podID="c2667584-632e-4394-80a2-175d8986ac4d" containerID="a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4" exitCode=0
Nov 02 21:20:06.445582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.445207    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-nkk94"
Nov 02 21:20:06.445582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.445252    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-nkk94" event={"ID":"c2667584-632e-4394-80a2-175d8986ac4d","Type":"ContainerDied","Data":"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"}
Nov 02 21:20:06.445582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.445303    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-nkk94" event={"ID":"c2667584-632e-4394-80a2-175d8986ac4d","Type":"ContainerDied","Data":"da8b9f1cf93f99497036d4a24dfa46bf889642d13c15949029eaf8cd90460f13"}
Nov 02 21:20:06.445582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.445329    4888 scope.go:122] "RemoveContainer" containerID="a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"
Nov 02 21:20:06.454023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.454003    4888 scope.go:122] "RemoveContainer" containerID="a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"
Nov 02 21:20:06.454306 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:20:06.454288    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4\": container with ID starting with a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4 not found: ID does not exist" containerID="a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"
Nov 02 21:20:06.454366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.454314    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4"} err="failed to get container status \"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4\": rpc error: code = NotFound desc = could not find container \"a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4\": container with ID starting with a03db5442407093365c1a4f5376a01a076c21967125ae6f5a06153d58fdac6c4 not found: ID does not exist"
Nov 02 21:20:06.603230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.603200    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:20:06.650762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:06.650729    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-nkk94"]
Nov 02 21:20:08.597936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:08.597899    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2667584-632e-4394-80a2-175d8986ac4d" path="/var/lib/kubelet/pods/c2667584-632e-4394-80a2-175d8986ac4d/volumes"
Nov 02 21:20:14.369730 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.369693    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"]
Nov 02 21:20:14.370244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.370091    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c2667584-632e-4394-80a2-175d8986ac4d" containerName="manager"
Nov 02 21:20:14.370244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.370104    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2667584-632e-4394-80a2-175d8986ac4d" containerName="manager"
Nov 02 21:20:14.370244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.370191    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2667584-632e-4394-80a2-175d8986ac4d" containerName="manager"
Nov 02 21:20:14.376673 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.376651    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456240    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456283    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6946c73-4dad-4e5b-987b-01def81ac372-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456299    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456314    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456427    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.456716 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.456571    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dbp\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-kube-api-access-q9dbp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.469866 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.469821    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"]
Nov 02 21:20:14.557953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.557919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dbp\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-kube-api-access-q9dbp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.557979    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.558000    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6946c73-4dad-4e5b-987b-01def81ac372-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.558015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.558031    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.558053    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.558170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.558077    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.559234 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.559200    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.561060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.561032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.565696 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.565642    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6946c73-4dad-4e5b-987b-01def81ac372-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.567616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.567592    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.568345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.568324    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6946c73-4dad-4e5b-987b-01def81ac372-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.613481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.613443    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.619157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.619129    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dbp\" (UniqueName: \"kubernetes.io/projected/d6946c73-4dad-4e5b-987b-01def81ac372-kube-api-access-q9dbp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-brtmx\" (UID: \"d6946c73-4dad-4e5b-987b-01def81ac372\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:14.688732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:14.688691    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:15.100805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.100782    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"]
Nov 02 21:20:15.102582 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:20:15.102555    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6946c73_4dad_4e5b_987b_01def81ac372.slice/crio-c4c553d2440d058ad30f42f405ca331094e80841cfd10ad478f65cf1fe63316e WatchSource:0}: Error finding container c4c553d2440d058ad30f42f405ca331094e80841cfd10ad478f65cf1fe63316e: Status 404 returned error can't find the container with id c4c553d2440d058ad30f42f405ca331094e80841cfd10ad478f65cf1fe63316e
Nov 02 21:20:15.105073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.105040    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:20:15.105162 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.105102    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:20:15.485836 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.485805    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx" event={"ID":"d6946c73-4dad-4e5b-987b-01def81ac372","Type":"ContainerStarted","Data":"0cf744b63b17bab7c9fcfc19336e4e349f0df8a7649b502a94b694efd3733dd6"}
Nov 02 21:20:15.486239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.485877    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx" event={"ID":"d6946c73-4dad-4e5b-987b-01def81ac372","Type":"ContainerStarted","Data":"c4c553d2440d058ad30f42f405ca331094e80841cfd10ad478f65cf1fe63316e"}
Nov 02 21:20:15.486239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.485903    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:15.596568 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:15.596516    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx" podStartSLOduration=1.596503111 podStartE2EDuration="1.596503111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:20:15.596196234 +0000 UTC m=+445.139988151" watchObservedRunningTime="2026-11-02 21:20:15.596503111 +0000 UTC m=+445.140295026"
Nov 02 21:20:16.493081 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:16.493049    4888 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:20:16.493458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:16.493109    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx" podUID="d6946c73-4dad-4e5b-987b-01def81ac372" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:20:19.491792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.491761    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx"
Nov 02 21:20:19.735367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.735330    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:20:19.735546 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.735525    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery" containerID="cri-o://a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb" gracePeriod=30
Nov 02 21:20:19.775648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.775571    4888 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-466l7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.27:8080/ready\": dial tcp 10.132.0.27:8080: connect: connection refused" start-of-body=
Nov 02 21:20:19.775648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.775626    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.27:8080/ready\": dial tcp 10.132.0.27:8080: connect: connection refused"
Nov 02 21:20:19.985224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:19.985192    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:20:20.106711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106623    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.106711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106676    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.106711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106708    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.107023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106779    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.107023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106829    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.107023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106881    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.107023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.106933    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\") pod \"4245ee27-1581-4c84-8f94-bc35b4f30622\" (UID: \"4245ee27-1581-4c84-8f94-bc35b4f30622\") "
Nov 02 21:20:20.107954 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.107796    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:20:20.111580 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.111549    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:20:20.111994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.111963    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "kube-api-access-zvhxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:20:20.112151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.112108    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:20:20.112357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.112329    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:20:20.112649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.112626    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:20:20.113271 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.113241    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert" pod "4245ee27-1581-4c84-8f94-bc35b4f30622" (UID: "4245ee27-1581-4c84-8f94-bc35b4f30622"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:20:20.208146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208110    4888 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-ca-configmap\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208146    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208160    4888 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4245ee27-1581-4c84-8f94-bc35b4f30622-local-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208172    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvhxs\" (UniqueName: \"kubernetes.io/projected/4245ee27-1581-4c84-8f94-bc35b4f30622-kube-api-access-zvhxs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208185    4888 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-csr-dns-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208198    4888 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-cacerts\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.208386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.208211    4888 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4245ee27-1581-4c84-8f94-bc35b4f30622-istio-kubeconfig\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:20:20.508896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.508864    4888 generic.go:356] "Generic (PLEG): container finished" podID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerID="a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb" exitCode=0
Nov 02 21:20:20.509300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.508917    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"
Nov 02 21:20:20.509300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.508933    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" event={"ID":"4245ee27-1581-4c84-8f94-bc35b4f30622","Type":"ContainerDied","Data":"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"}
Nov 02 21:20:20.509300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.508996    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7" event={"ID":"4245ee27-1581-4c84-8f94-bc35b4f30622","Type":"ContainerDied","Data":"3c0e6db97b7f6e4b2c62b10e907b21ad135ba7fa4747c2a5990261891578a086"}
Nov 02 21:20:20.509300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.509015    4888 scope.go:122] "RemoveContainer" containerID="a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"
Nov 02 21:20:20.518544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.518523    4888 scope.go:122] "RemoveContainer" containerID="a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"
Nov 02 21:20:20.518795 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:20:20.518776    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb\": container with ID starting with a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb not found: ID does not exist" containerID="a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"
Nov 02 21:20:20.518835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.518806    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb"} err="failed to get container status \"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb\": rpc error: code = NotFound desc = could not find container \"a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb\": container with ID starting with a0b16dac2972949865d440ba668cbdab75d20781bc0a0266199f5331cd8415eb not found: ID does not exist"
Nov 02 21:20:20.664770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.664740    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:20:20.668943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.668910    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7"]
Nov 02 21:20:20.728235 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.728206    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-w2wvv"]
Nov 02 21:20:20.728544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.728532    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery"
Nov 02 21:20:20.728592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.728545    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery"
Nov 02 21:20:20.728624 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.728611    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" containerName="discovery"
Nov 02 21:20:20.733223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.733204    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.784590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.784506    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 02 21:20:20.784590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.784529    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nlvhp\""
Nov 02 21:20:20.785762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.785735    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-w2wvv"]
Nov 02 21:20:20.814108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.814081    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prnl9\" (UniqueName: \"kubernetes.io/projected/68674783-8db7-4e17-8850-6da80b9259d8-kube-api-access-prnl9\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.814279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.814136    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68674783-8db7-4e17-8850-6da80b9259d8-cert\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.845365 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.845332    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:20:20.848953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.848933    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:20.900749 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.900700    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 02 21:20:20.900749 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.900749    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cmkqt\""
Nov 02 21:20:20.902229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.902204    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:20:20.915231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.915210    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68674783-8db7-4e17-8850-6da80b9259d8-cert\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.915337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.915259    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tq4t\" (UniqueName: \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:20.915414 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.915390    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prnl9\" (UniqueName: \"kubernetes.io/projected/68674783-8db7-4e17-8850-6da80b9259d8-kube-api-access-prnl9\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.915473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.915454    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:20.917615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.917594    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68674783-8db7-4e17-8850-6da80b9259d8-cert\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:20.971838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:20.971796    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prnl9\" (UniqueName: \"kubernetes.io/projected/68674783-8db7-4e17-8850-6da80b9259d8-kube-api-access-prnl9\") pod \"kserve-controller-manager-776cc66d47-w2wvv\" (UID: \"68674783-8db7-4e17-8850-6da80b9259d8\") " pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:21.016222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.016183    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:21.016413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.016288    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tq4t\" (UniqueName: \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:21.018821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.018795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:21.045916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.045818    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:21.077024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.076977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tq4t\" (UniqueName: \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\") pod \"llmisvc-controller-manager-869d5dcc5d-fd4dg\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:21.161376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.161336    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:21.412036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.412010    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-w2wvv"]
Nov 02 21:20:21.414109 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:20:21.414075    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68674783_8db7_4e17_8850_6da80b9259d8.slice/crio-4eb61f91452ce3dbce0ba99d0c7dbb8eb9e5fe1e0071df350f48c66e8c3d5aa3 WatchSource:0}: Error finding container 4eb61f91452ce3dbce0ba99d0c7dbb8eb9e5fe1e0071df350f48c66e8c3d5aa3: Status 404 returned error can't find the container with id 4eb61f91452ce3dbce0ba99d0c7dbb8eb9e5fe1e0071df350f48c66e8c3d5aa3
Nov 02 21:20:21.516097 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.516060    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv" event={"ID":"68674783-8db7-4e17-8850-6da80b9259d8","Type":"ContainerStarted","Data":"4eb61f91452ce3dbce0ba99d0c7dbb8eb9e5fe1e0071df350f48c66e8c3d5aa3"}
Nov 02 21:20:21.522500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:21.522476    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:20:21.524599 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:20:21.524571    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podca553095_6048_49ac_a029_016c227f91b4.slice/crio-26d1f2524b6d489b889658ce6c0c28cb588b6e50036d9815cd6f203a7e273072 WatchSource:0}: Error finding container 26d1f2524b6d489b889658ce6c0c28cb588b6e50036d9815cd6f203a7e273072: Status 404 returned error can't find the container with id 26d1f2524b6d489b889658ce6c0c28cb588b6e50036d9815cd6f203a7e273072
Nov 02 21:20:22.258558 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:20:22.258517    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:20:22.523343 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:22.523269    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" event={"ID":"ca553095-6048-49ac-a029-016c227f91b4","Type":"ContainerStarted","Data":"26d1f2524b6d489b889658ce6c0c28cb588b6e50036d9815cd6f203a7e273072"}
Nov 02 21:20:22.600697 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:22.600660    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4245ee27-1581-4c84-8f94-bc35b4f30622" path="/var/lib/kubelet/pods/4245ee27-1581-4c84-8f94-bc35b4f30622/volumes"
Nov 02 21:20:24.539418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.539382    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" event={"ID":"ca553095-6048-49ac-a029-016c227f91b4","Type":"ContainerStarted","Data":"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"}
Nov 02 21:20:24.539887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.539486    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:24.541313 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.541290    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv" event={"ID":"68674783-8db7-4e17-8850-6da80b9259d8","Type":"ContainerStarted","Data":"829d2798c5f0c513a133af5c1d49310f8c02fd5f29635cd591fe7d2f894ccd2e"}
Nov 02 21:20:24.541443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.541427    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:20:24.647168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.647116    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" podStartSLOduration=2.075555442 podStartE2EDuration="4.647102099s" totalImagesPullingTime="2.571546657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:20:24.646309553 +0000 UTC m=+454.190101469" watchObservedRunningTime="2026-11-02 21:20:24.647102099 +0000 UTC m=+454.190894015"
Nov 02 21:20:24.757098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:24.757043    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv" podStartSLOduration=2.137830435 podStartE2EDuration="4.757028424s" totalImagesPullingTime="2.619197989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:20:24.756065309 +0000 UTC m=+454.299857236" watchObservedRunningTime="2026-11-02 21:20:24.757028424 +0000 UTC m=+454.300820336"
Nov 02 21:20:40.089990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.089953    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 21:20:40.093524 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.093500    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.190439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.190400    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.190609 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.190525    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.190683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.190597    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgxw7\" (UniqueName: \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.190683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.190641    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.192530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.192506    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 21:20:40.192650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.192555    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 21:20:40.192650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.192588    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 21:20:40.192755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.192690    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 21:20:40.192755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.192711    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-47p47\""
Nov 02 21:20:40.193930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.193911    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 21:20:40.291776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.291735    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgxw7\" (UniqueName: \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.291776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.291780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.292019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.291978    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.292107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.292077    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.292481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.292458    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.294518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.294492    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.294629 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.294545    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.346146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.346071    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgxw7\" (UniqueName: \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\") pod \"maas-api-5d98fc7974-fqwl4\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") " pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.406015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.405976    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:40.774193 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:40.774167    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 21:20:40.776342 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:20:40.776306    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a1e0d0_7373_4eea_8e6a_cb5f4ece5757.slice/crio-ec85b056a5bfee6582d108500921a1dbebfb6df5f086d0f97618b4c1e976dceb WatchSource:0}: Error finding container ec85b056a5bfee6582d108500921a1dbebfb6df5f086d0f97618b4c1e976dceb: Status 404 returned error can't find the container with id ec85b056a5bfee6582d108500921a1dbebfb6df5f086d0f97618b4c1e976dceb
Nov 02 21:20:41.615387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:41.615351    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" event={"ID":"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757","Type":"ContainerStarted","Data":"ec85b056a5bfee6582d108500921a1dbebfb6df5f086d0f97618b4c1e976dceb"}
Nov 02 21:20:43.625861 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:43.625803    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" event={"ID":"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757","Type":"ContainerStarted","Data":"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"}
Nov 02 21:20:43.626278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:43.625948    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:43.736561 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:43.736510    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" podStartSLOduration=1.7944208659999998 podStartE2EDuration="3.736496374s" totalImagesPullingTime="1.942075508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:20:43.734938479 +0000 UTC m=+473.278730593" watchObservedRunningTime="2026-11-02 21:20:43.736496374 +0000 UTC m=+473.280288284"
Nov 02 21:20:49.635455 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:49.635420    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 21:20:52.259915 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:20:52.259874    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:20:55.548522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:55.548486    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:20:55.551595 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:20:55.551568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-w2wvv"
Nov 02 21:21:22.258936 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:21:22.258828    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:21:44.830121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:44.830079    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"]
Nov 02 21:21:44.833819 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:44.833795    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:44.885618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:44.885587    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"]
Nov 02 21:21:44.944106 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:44.944069    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96b5de50-e2ab-438e-af63-984d3cb0e653-cert\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:44.944106 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:44.944109    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rshqr\" (UniqueName: \"kubernetes.io/projected/96b5de50-e2ab-438e-af63-984d3cb0e653-kube-api-access-rshqr\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.044618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.044575    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96b5de50-e2ab-438e-af63-984d3cb0e653-cert\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.044618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.044626    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rshqr\" (UniqueName: \"kubernetes.io/projected/96b5de50-e2ab-438e-af63-984d3cb0e653-kube-api-access-rshqr\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.047254 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.047228    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96b5de50-e2ab-438e-af63-984d3cb0e653-cert\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.098966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.098880    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rshqr\" (UniqueName: \"kubernetes.io/projected/96b5de50-e2ab-438e-af63-984d3cb0e653-kube-api-access-rshqr\") pod \"llmisvc-controller-manager-6466c554d8-7r2zx\" (UID: \"96b5de50-e2ab-438e-af63-984d3cb0e653\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.145879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.145807    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:45.506671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.506647    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"]
Nov 02 21:21:45.508861 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:21:45.508818    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b5de50_e2ab_438e_af63_984d3cb0e653.slice/crio-aa2ff4ba04611eaa3df9f96030729ab41dfba573e68f736067d45bc1cc27fa8e WatchSource:0}: Error finding container aa2ff4ba04611eaa3df9f96030729ab41dfba573e68f736067d45bc1cc27fa8e: Status 404 returned error can't find the container with id aa2ff4ba04611eaa3df9f96030729ab41dfba573e68f736067d45bc1cc27fa8e
Nov 02 21:21:45.913125 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:45.913083    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx" event={"ID":"96b5de50-e2ab-438e-af63-984d3cb0e653","Type":"ContainerStarted","Data":"aa2ff4ba04611eaa3df9f96030729ab41dfba573e68f736067d45bc1cc27fa8e"}
Nov 02 21:21:46.919972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:46.919933    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx" event={"ID":"96b5de50-e2ab-438e-af63-984d3cb0e653","Type":"ContainerStarted","Data":"b3bdd1b84aeb573d983ef1aadbb750c533ecf744fb2ec2a82eba6841e99e327b"}
Nov 02 21:21:46.920396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:46.920078    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:21:47.030261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:21:47.030204    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx" podStartSLOduration=2.209207939 podStartE2EDuration="3.030190225s" totalImagesPullingTime="820.982286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:21:47.028697115 +0000 UTC m=+536.572489032" watchObservedRunningTime="2026-11-02 21:21:47.030190225 +0000 UTC m=+536.573982149"
Nov 02 21:21:52.258406 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:21:52.258370    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:22:17.926701 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:17.926671    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7r2zx"
Nov 02 21:22:18.082890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.082842    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:22:18.083186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.083149    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" podUID="ca553095-6048-49ac-a029-016c227f91b4" containerName="manager" containerID="cri-o://b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a" gracePeriod=30
Nov 02 21:22:18.345479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.345450    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:22:18.420671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.420634    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\" (UniqueName: \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\") pod \"ca553095-6048-49ac-a029-016c227f91b4\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") "
Nov 02 21:22:18.420843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.420728    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\" (UniqueName: \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\") pod \"ca553095-6048-49ac-a029-016c227f91b4\" (UID: \"ca553095-6048-49ac-a029-016c227f91b4\") "
Nov 02 21:22:18.422712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.422680    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t" pod "ca553095-6048-49ac-a029-016c227f91b4" (UID: "ca553095-6048-49ac-a029-016c227f91b4"). InnerVolumeSpecName "kube-api-access-7tq4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:22:18.422797 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.422770    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert" pod "ca553095-6048-49ac-a029-016c227f91b4" (UID: "ca553095-6048-49ac-a029-016c227f91b4"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:22:18.521794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.521702    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tq4t\" (UniqueName: \"kubernetes.io/projected/ca553095-6048-49ac-a029-016c227f91b4-kube-api-access-7tq4t\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:22:18.521794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:18.521739    4888 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca553095-6048-49ac-a029-016c227f91b4-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:22:19.062181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.062149    4888 generic.go:356] "Generic (PLEG): container finished" podID="ca553095-6048-49ac-a029-016c227f91b4" containerID="b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a" exitCode=0
Nov 02 21:22:19.062601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.062196    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"
Nov 02 21:22:19.062601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.062198    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" event={"ID":"ca553095-6048-49ac-a029-016c227f91b4","Type":"ContainerDied","Data":"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"}
Nov 02 21:22:19.062601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.062323    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg" event={"ID":"ca553095-6048-49ac-a029-016c227f91b4","Type":"ContainerDied","Data":"26d1f2524b6d489b889658ce6c0c28cb588b6e50036d9815cd6f203a7e273072"}
Nov 02 21:22:19.062601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.062356    4888 scope.go:122] "RemoveContainer" containerID="b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"
Nov 02 21:22:19.071188 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.071165    4888 scope.go:122] "RemoveContainer" containerID="b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"
Nov 02 21:22:19.071492 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:22:19.071471    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a\": container with ID starting with b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a not found: ID does not exist" containerID="b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"
Nov 02 21:22:19.071551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.071503    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a"} err="failed to get container status \"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a\": rpc error: code = NotFound desc = could not find container \"b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a\": container with ID starting with b32f3980ebaa37537869ecb337f1d88b9f21b5a66ff5d837e415722824db8b8a not found: ID does not exist"
Nov 02 21:22:19.221076 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.221035    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:22:19.268399 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:19.268362    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-fd4dg"]
Nov 02 21:22:20.599086 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:20.599045    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca553095-6048-49ac-a029-016c227f91b4" path="/var/lib/kubelet/pods/ca553095-6048-49ac-a029-016c227f91b4/volumes"
Nov 02 21:22:22.259717 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:22:22.259612    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:22:24.085958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:24.085923    4888 generic.go:356] "Generic (PLEG): container finished" podID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerID="f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852" exitCode=1
Nov 02 21:22:24.086366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:24.085992    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerDied","Data":"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"}
Nov 02 21:22:24.086366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:24.086337    4888 scope.go:122] "RemoveContainer" containerID="f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"
Nov 02 21:22:25.396911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:25.396773    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:22:26.098378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:26.098343    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerStarted","Data":"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"}
Nov 02 21:22:26.098562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:26.098411    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:22:37.106508 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:37.106433    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 21:22:50.650629 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:50.650597    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:22:50.652762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:22:50.652736    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:22:52.258259 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:22:52.258219    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:23:07.858539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.858500    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:23:07.859045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.859025    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ca553095-6048-49ac-a029-016c227f91b4" containerName="manager"
Nov 02 21:23:07.859099 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.859048    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca553095-6048-49ac-a029-016c227f91b4" containerName="manager"
Nov 02 21:23:07.859160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.859149    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca553095-6048-49ac-a029-016c227f91b4" containerName="manager"
Nov 02 21:23:07.862275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.862247    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.916479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.916438    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-445mt\""
Nov 02 21:23:07.953014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.952976    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953028    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953045    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953170    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953288 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953203    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953288 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953301    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgv4l\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:07.953430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:07.953406    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054197 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054159    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054197 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054200    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054233    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054260    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054280    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054300    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgv4l\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054320    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054347    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054384    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054639    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054707    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.054990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.054967    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.055054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.055038    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.055301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.055276    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.056663 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.056643    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.057150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.057117    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.061416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.061387    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:23:08.108516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.108481    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.108724 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.108678    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgv4l\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-tcck7\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.175587 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.175552    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:08.589515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:08.589486    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:23:08.591399 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:23:08.591371    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8492a874_e5b6_46a9_9d09_cc75809f58e8.slice/crio-f9a2df12afbc79068f9a11e0425be71b05dee1a0e3ac4a9f2cd4d42e8aa0fcee WatchSource:0}: Error finding container f9a2df12afbc79068f9a11e0425be71b05dee1a0e3ac4a9f2cd4d42e8aa0fcee: Status 404 returned error can't find the container with id f9a2df12afbc79068f9a11e0425be71b05dee1a0e3ac4a9f2cd4d42e8aa0fcee
Nov 02 21:23:09.302787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:09.302749    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" event={"ID":"8492a874-e5b6-46a9-9d09-cc75809f58e8","Type":"ContainerStarted","Data":"f9a2df12afbc79068f9a11e0425be71b05dee1a0e3ac4a9f2cd4d42e8aa0fcee"}
Nov 02 21:23:12.808397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:12.808358    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:23:12.808718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:12.808425    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:23:12.808718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:12.808454    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:23:13.337901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:13.337408    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" event={"ID":"8492a874-e5b6-46a9-9d09-cc75809f58e8","Type":"ContainerStarted","Data":"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"}
Nov 02 21:23:13.548244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:13.548162    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podStartSLOduration=2.334109167 podStartE2EDuration="6.548140882s" totalImagesPullingTime="4.214031715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:23:13.499741006 +0000 UTC m=+623.043532923" watchObservedRunningTime="2026-11-02 21:23:13.548140882 +0000 UTC m=+623.091932799"
Nov 02 21:23:14.176028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:14.175977    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:14.177741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:14.177707    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:14.177890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:14.177785    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:15.176381 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:15.176345    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:15.176781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:15.176401    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:16.176871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:16.176820    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:16.177341 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:16.176890    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:17.177018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:17.176977    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:17.177423 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:17.177037    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:18.176117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:18.176081    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:18.176305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:18.176139    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:19.176756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:19.176712    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:19.177172 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:19.176778    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:20.176637 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:20.176593    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:23:20.176820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:20.176664    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Nov 02 21:23:21.180147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:21.180112    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:21.180571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:21.180384    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:21.180992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:23:21.180975    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:23:22.260914 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:23:22.260877    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:23:52.259253 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:23:52.259208    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:24:22.259315 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:22.259234    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:24:38.309289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.309243    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:24:38.313000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.312975    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.369472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.369422    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.369674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.369498    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.369674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.369561    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-665lp\" (UniqueName: \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.369674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.369591    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.405501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.405463    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:24:38.413735 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.413708    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-ef6b43-serving-cert\""
Nov 02 21:24:38.470128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.470090    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.470318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.470143    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.470318 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:38.470249    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-serving-cert: secret "maas-api-e2e-worker-w5-ef6b43-serving-cert" not found
Nov 02 21:24:38.470318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.470259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-665lp\" (UniqueName: \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.470318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.470297    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.470318 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:38.470319    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls podName:990bdd1c-ddd7-4025-a57c-8dc0aa083950 nodeName:}" failed. No retries permitted until 2026-11-02 21:24:38.970300377 +0000 UTC m=+708.514092289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls") pod "maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" (UID: "990bdd1c-ddd7-4025-a57c-8dc0aa083950") : secret "maas-api-e2e-worker-w5-ef6b43-serving-cert" not found
Nov 02 21:24:38.470802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.470784    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.472776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.472751    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.527007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.526974    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-665lp\" (UniqueName: \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.974606 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.974569    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:38.977039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:38.977010    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:39.227176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:39.227091    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:39.596473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:39.596444    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:24:39.598598 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:24:39.598566    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990bdd1c_ddd7_4025_a57c_8dc0aa083950.slice/crio-f7312b1455e87fd42a7c8a9a100bf4c2fb6773f1ef9d2a6106c2bcc4d2142f6e WatchSource:0}: Error finding container f7312b1455e87fd42a7c8a9a100bf4c2fb6773f1ef9d2a6106c2bcc4d2142f6e: Status 404 returned error can't find the container with id f7312b1455e87fd42a7c8a9a100bf4c2fb6773f1ef9d2a6106c2bcc4d2142f6e
Nov 02 21:24:39.600044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:39.600028    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:24:39.739418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:39.739383    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" event={"ID":"990bdd1c-ddd7-4025-a57c-8dc0aa083950","Type":"ContainerStarted","Data":"f7312b1455e87fd42a7c8a9a100bf4c2fb6773f1ef9d2a6106c2bcc4d2142f6e"}
Nov 02 21:24:40.745109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:40.745063    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" event={"ID":"990bdd1c-ddd7-4025-a57c-8dc0aa083950","Type":"ContainerStarted","Data":"ca4f0e7ff381e87bf36b155c1267b6445c90ccd3f8cd8acb7770857af92bb3b6"}
Nov 02 21:24:40.745598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:40.745189    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:40.853818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:40.853715    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" podStartSLOduration=1.9178420699999998 podStartE2EDuration="2.853693611s" totalImagesPullingTime="935.851541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:24:40.853078881 +0000 UTC m=+710.396870799" watchObservedRunningTime="2026-11-02 21:24:40.853693611 +0000 UTC m=+710.397485526"
Nov 02 21:24:46.754454 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:46.754423    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:24:50.225536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.225497    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 21:24:50.229121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.229094    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.281121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.281091    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 21:24:50.289295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.289262    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5fz\" (UniqueName: \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.289449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.289313    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.326616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.326584    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-284b8e\""
Nov 02 21:24:50.326946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.326927    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-284b8e-dockercfg-6m955\""
Nov 02 21:24:50.390801 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.390760    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5fz\" (UniqueName: \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.391004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.390811    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.391570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.391549    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.445083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.445051    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5fz\" (UniqueName: \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\") pod \"payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.543397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.543045    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:50.913544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:50.913520    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 21:24:50.915805 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:24:50.915775    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9df833_90ea_437b_aa0a_a23e7a181bb6.slice/crio-e263e2d7afbe5bbb127c219e35c1d60f26727cc862bf7daf1b3c1cf93291fa22 WatchSource:0}: Error finding container e263e2d7afbe5bbb127c219e35c1d60f26727cc862bf7daf1b3c1cf93291fa22: Status 404 returned error can't find the container with id e263e2d7afbe5bbb127c219e35c1d60f26727cc862bf7daf1b3c1cf93291fa22
Nov 02 21:24:51.798458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:51.798423    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" event={"ID":"ef9df833-90ea-437b-aa0a-a23e7a181bb6","Type":"ContainerStarted","Data":"e263e2d7afbe5bbb127c219e35c1d60f26727cc862bf7daf1b3c1cf93291fa22"}
Nov 02 21:24:52.263792 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:52.263757    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:24:53.072945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.072906    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 21:24:53.079568 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.079543    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.126693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.126659    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 21:24:53.171943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.171907    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-284b8e-serving-cert\""
Nov 02 21:24:53.215538 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.215503    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.215723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.215553    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.215723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.215620    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.215723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.215642    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.318417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.316982    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.318417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.317034    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.318417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.317152    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.318417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.317180    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.318417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.318035    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.319181 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:53.319151    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-serving-cert: secret "maas-api-e2e-worker-w6-284b8e-serving-cert" not found
Nov 02 21:24:53.319314 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:24:53.319216    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls podName:6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed nodeName:}" failed. No retries permitted until 2026-11-02 21:24:53.819197663 +0000 UTC m=+723.362989573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls") pod "maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" (UID: "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed") : secret "maas-api-e2e-worker-w6-284b8e-serving-cert" not found
Nov 02 21:24:53.322254 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.322230    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.376892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.376801    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.822406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.822365    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.825060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.825034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") pod \"maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:53.995715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:53.995678    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:24:54.365259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:54.365232    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 21:24:54.367065 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:24:54.367034    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6974c5c3_ffb3_45f6_98c9_ff0cbb2c70ed.slice/crio-12abff4cc4c8af8b3a4a2a6d87f9ebdd2a971b4a6984b5d03dd6b96417ed788a WatchSource:0}: Error finding container 12abff4cc4c8af8b3a4a2a6d87f9ebdd2a971b4a6984b5d03dd6b96417ed788a: Status 404 returned error can't find the container with id 12abff4cc4c8af8b3a4a2a6d87f9ebdd2a971b4a6984b5d03dd6b96417ed788a
Nov 02 21:24:54.814966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:54.814927    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" event={"ID":"ef9df833-90ea-437b-aa0a-a23e7a181bb6","Type":"ContainerStarted","Data":"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"}
Nov 02 21:24:54.815304 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:54.815028    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:24:54.816219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:54.816187    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" event={"ID":"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed","Type":"ContainerStarted","Data":"12abff4cc4c8af8b3a4a2a6d87f9ebdd2a971b4a6984b5d03dd6b96417ed788a"}
Nov 02 21:24:54.924357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:54.924299    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" podStartSLOduration=1.945437318 podStartE2EDuration="4.924285861s" totalImagesPullingTime="2.978848543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:24:54.922336694 +0000 UTC m=+724.466128613" watchObservedRunningTime="2026-11-02 21:24:54.924285861 +0000 UTC m=+724.468077776"
Nov 02 21:24:55.822533 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:55.822493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" event={"ID":"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed","Type":"ContainerStarted","Data":"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"}
Nov 02 21:24:55.823057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:24:55.822581    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:25:01.833016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:01.832981    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 21:25:01.942377 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:01.942321    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" podStartSLOduration=8.017404872 podStartE2EDuration="8.942296172s" totalImagesPullingTime="924.8913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:24:55.931282692 +0000 UTC m=+725.475074609" watchObservedRunningTime="2026-11-02 21:25:01.942296172 +0000 UTC m=+731.486088084"
Nov 02 21:25:05.826539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:05.826504    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 21:25:06.960576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:06.960538    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:25:06.964435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:06.964413    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.015245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.015202    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-dd6574-serving-cert\""
Nov 02 21:25:07.047323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.047283    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.047323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.047335    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.047516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.047410    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skvmc\" (UniqueName: \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.047556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.047511    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.064362 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.064332    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:25:07.148502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.148467    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.148687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.148535    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skvmc\" (UniqueName: \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.148687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.148579    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.148687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.148640    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.148881 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:25:07.148743    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-serving-cert: secret "maas-api-e2e-worker-w0-dd6574-serving-cert" not found
Nov 02 21:25:07.148881 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:25:07.148818    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls podName:0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe nodeName:}" failed. No retries permitted until 2026-11-02 21:25:07.648801847 +0000 UTC m=+737.192593745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls") pod "maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" (UID: "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe") : secret "maas-api-e2e-worker-w0-dd6574-serving-cert" not found
Nov 02 21:25:07.149308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.149286    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.150934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.150916    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.206385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.203946    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skvmc\" (UniqueName: \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.652981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.652941    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.655490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.655464    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:07.878286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:07.878253    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:08.244723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:08.244699    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:25:08.246662 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:25:08.246631    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f86cd50_af04_43bd_b5b9_9e1ad99c3dbe.slice/crio-c69b4c34ac0d3d7713dfcb6e3cb3d735267cec0804260a92d01fb667fde81a74 WatchSource:0}: Error finding container c69b4c34ac0d3d7713dfcb6e3cb3d735267cec0804260a92d01fb667fde81a74: Status 404 returned error can't find the container with id c69b4c34ac0d3d7713dfcb6e3cb3d735267cec0804260a92d01fb667fde81a74
Nov 02 21:25:08.879632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:08.879599    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" event={"ID":"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe","Type":"ContainerStarted","Data":"c69b4c34ac0d3d7713dfcb6e3cb3d735267cec0804260a92d01fb667fde81a74"}
Nov 02 21:25:09.886082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:09.886046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" event={"ID":"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe","Type":"ContainerStarted","Data":"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"}
Nov 02 21:25:09.886475 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:09.886154    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:09.994700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:09.994644    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" podStartSLOduration=3.131159586 podStartE2EDuration="3.99462909s" totalImagesPullingTime="863.469504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:25:09.993686693 +0000 UTC m=+739.537478606" watchObservedRunningTime="2026-11-02 21:25:09.99462909 +0000 UTC m=+739.538421001"
Nov 02 21:25:15.896310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:15.896279    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:25:22.258239 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:25:22.258201    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:25:48.791484 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.791448    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 21:25:48.795344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.795324    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.893053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.892971    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4d9ea0\"/\"openshift-service-ca.crt\""
Nov 02 21:25:48.893053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.893014    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4d9ea0\"/\"kube-root-ca.crt\""
Nov 02 21:25:48.893301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.893175    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4d9ea0\"/\"simulator-w2-4d9ea0-kserve-self-signed-certs\""
Nov 02 21:25:48.893301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.893213    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4d9ea0\"/\"default-dockercfg-fpf4h\""
Nov 02 21:25:48.894882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.894839    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 21:25:48.958430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958398    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.958622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958435    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.958622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958461    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.958622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958571    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.958754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958667    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:48.958754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:48.958693    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpq4\" (UniqueName: \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.059713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059617    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.059713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059674    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.059713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059708    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059728    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpq4\" (UniqueName: \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059790    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.059812    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.060145    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.060178    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.060305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.060246    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.062208 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.062185    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.062429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.062410    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.115409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.115368    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpq4\" (UniqueName: \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\") pod \"simulator-w2-4d9ea0-kserve-589987c764-fjm4j\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") " pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.414198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.414146    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:49.781712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:49.781685    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 21:25:49.783548 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:25:49.783525    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d894546_4031_4d92_80eb_7d5f60fb59c1.slice/crio-bdf78b8e8860f252d8303299445151e563b0b3e9f5187721a42a0dd52b412787 WatchSource:0}: Error finding container bdf78b8e8860f252d8303299445151e563b0b3e9f5187721a42a0dd52b412787: Status 404 returned error can't find the container with id bdf78b8e8860f252d8303299445151e563b0b3e9f5187721a42a0dd52b412787
Nov 02 21:25:50.082777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:50.082703    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" event={"ID":"5d894546-4031-4d92-80eb-7d5f60fb59c1","Type":"ContainerStarted","Data":"bdf78b8e8860f252d8303299445151e563b0b3e9f5187721a42a0dd52b412787"}
Nov 02 21:25:52.260589 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:25:52.260547    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:25:53.099458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:53.099425    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" event={"ID":"5d894546-4031-4d92-80eb-7d5f60fb59c1","Type":"ContainerStarted","Data":"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"}
Nov 02 21:25:53.414412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:53.414374    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:59.414706 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:59.414661    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:59.427037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:59.427001    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:25:59.535470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:25:59.535420    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podStartSLOduration=9.242016238 podStartE2EDuration="11.535408121s" totalImagesPullingTime="2.293391883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:25:53.22297063 +0000 UTC m=+782.766762547" watchObservedRunningTime="2026-11-02 21:25:59.535408121 +0000 UTC m=+789.079200037"
Nov 02 21:26:00.142008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:00.141976    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:26:06.338319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.338282    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 21:26:06.342032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.342008    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417489    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417536    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417559    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417656    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417686    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.417725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.417703    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx6mr\" (UniqueName: \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.440171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.440131    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-284b8e\"/\"simulator-w6-284b8e-kserve-self-signed-certs\""
Nov 02 21:26:06.440171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.440156    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-284b8e\"/\"default-dockercfg-kqqs4\""
Nov 02 21:26:06.440171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.440180    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-284b8e\"/\"openshift-service-ca.crt\""
Nov 02 21:26:06.440426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.440267    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-284b8e\"/\"kube-root-ca.crt\""
Nov 02 21:26:06.441263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.441243    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 21:26:06.519218 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519182    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519227    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519288    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519317    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519342    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx6mr\" (UniqueName: \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519663 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519640    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519668    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.519781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.519766    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.521588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.521570    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.521896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.521877    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.573772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.573736    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx6mr\" (UniqueName: \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\") pod \"simulator-w6-284b8e-kserve-7669b96c64-sd5xh\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") " pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:06.653553 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:06.653520    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:07.024380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:07.024353    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 21:26:07.025877 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:26:07.025827    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc96e38d2_104d_4fe7_bc6d_6407dc38acca.slice/crio-6f594a5d6f299eae01f7fe41c36d0ef7c23848d15c3b266eb9605bfe9770fda7 WatchSource:0}: Error finding container 6f594a5d6f299eae01f7fe41c36d0ef7c23848d15c3b266eb9605bfe9770fda7: Status 404 returned error can't find the container with id 6f594a5d6f299eae01f7fe41c36d0ef7c23848d15c3b266eb9605bfe9770fda7
Nov 02 21:26:07.164799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:07.164761    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" event={"ID":"c96e38d2-104d-4fe7-bc6d-6407dc38acca","Type":"ContainerStarted","Data":"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"}
Nov 02 21:26:07.165004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:07.164807    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" event={"ID":"c96e38d2-104d-4fe7-bc6d-6407dc38acca","Type":"ContainerStarted","Data":"6f594a5d6f299eae01f7fe41c36d0ef7c23848d15c3b266eb9605bfe9770fda7"}
Nov 02 21:26:07.274236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:07.274136    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podStartSLOduration=1.274119032 podStartE2EDuration="1.274119032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:26:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:26:07.272934164 +0000 UTC m=+796.816726080" watchObservedRunningTime="2026-11-02 21:26:07.274119032 +0000 UTC m=+796.817910947"
Nov 02 21:26:07.654020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:07.653987    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:16.653756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:16.653716    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:16.666816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:16.666788    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:17.223198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:17.223161    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:26:22.259232 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:26:22.259194    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:26:25.810109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.810072    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:26:25.816518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.816493    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.866311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.866277    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:26:25.912010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.911975    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ef6b43\"/\"kube-root-ca.crt\""
Nov 02 21:26:25.912010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.911996    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ef6b43\"/\"default-dockercfg-fgsj4\""
Nov 02 21:26:25.912010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.911975    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ef6b43\"/\"openshift-service-ca.crt\""
Nov 02 21:26:25.913540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.913524    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ef6b43\"/\"premium-w5-ef6b43-kserve-self-signed-certs\""
Nov 02 21:26:25.914916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.914895    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.914986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.914925    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.914986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.914973    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9wpn\" (UniqueName: \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.915072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.915057    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.915163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.915147    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:25.915211 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:25.915184    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016559    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016597    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016621    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9wpn\" (UniqueName: \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016649    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016696    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.016962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.016814    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.017240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.017214    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.017240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.017231    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.017395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.017322    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.019018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.018995    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.019175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.019156    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.070701 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.070630    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9wpn\" (UniqueName: \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\") pod \"premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") " pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.130293 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.130249    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:26.498177 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:26.498144    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:26:26.499884 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:26:26.499840    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f32883b_8fc9_404a_8c2c_774c83ca5d70.slice/crio-b233521315588309d5b118cf2275a5d659e5f957a91bd703953282c546e10b83 WatchSource:0}: Error finding container b233521315588309d5b118cf2275a5d659e5f957a91bd703953282c546e10b83: Status 404 returned error can't find the container with id b233521315588309d5b118cf2275a5d659e5f957a91bd703953282c546e10b83
Nov 02 21:26:27.261464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:27.261426    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" event={"ID":"9f32883b-8fc9-404a-8c2c-774c83ca5d70","Type":"ContainerStarted","Data":"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"}
Nov 02 21:26:27.261464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:27.261470    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" event={"ID":"9f32883b-8fc9-404a-8c2c-774c83ca5d70","Type":"ContainerStarted","Data":"b233521315588309d5b118cf2275a5d659e5f957a91bd703953282c546e10b83"}
Nov 02 21:26:27.369687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:27.369633    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podStartSLOduration=2.36962036 podStartE2EDuration="2.36962036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:26:27.369025503 +0000 UTC m=+816.912817416" watchObservedRunningTime="2026-11-02 21:26:27.36962036 +0000 UTC m=+816.913412275"
Nov 02 21:26:28.131227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:28.131185    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:36.130577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:36.130539    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:36.143607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:36.143569    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:36.315922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:36.315894    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:26:38.653861 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.653821    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 21:26:38.657265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.657234    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.747017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.746974    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 21:26:38.756072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.756036    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9a6cf2\"/\"kube-root-ca.crt\""
Nov 02 21:26:38.756072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.756055    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9a6cf2\"/\"openshift-service-ca.crt\""
Nov 02 21:26:38.756270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.756097    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9a6cf2\"/\"simulator-w3-9a6cf2-kserve-self-signed-certs\""
Nov 02 21:26:38.756270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.756139    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9a6cf2\"/\"default-dockercfg-ptsrn\""
Nov 02 21:26:38.838948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.838885    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnh8\" (UniqueName: \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.839103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.839013    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.839103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.839056    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.839103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.839093    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.839228 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.839108    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.839297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.839281    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940329    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnh8\" (UniqueName: \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940368    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940392    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940436    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940457    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940530    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940968    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.940995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.940986    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.941237 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.941218    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.942827 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.942795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.943117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.943101    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:38.995903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:38.995872    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnh8\" (UniqueName: \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\") pod \"simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") " pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:39.270026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:39.269943    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:39.635715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:39.635685    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 21:26:39.637789 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:26:39.637762    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9990587_21c7_4653_ba57_46a3be01d996.slice/crio-741f7d3f9e413977b74787d4dd72d1a0140c4c1779e93b5304a27a11ac7fd51e WatchSource:0}: Error finding container 741f7d3f9e413977b74787d4dd72d1a0140c4c1779e93b5304a27a11ac7fd51e: Status 404 returned error can't find the container with id 741f7d3f9e413977b74787d4dd72d1a0140c4c1779e93b5304a27a11ac7fd51e
Nov 02 21:26:40.327829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:40.327790    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" event={"ID":"a9990587-21c7-4653-ba57-46a3be01d996","Type":"ContainerStarted","Data":"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"}
Nov 02 21:26:40.328261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:40.327842    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" event={"ID":"a9990587-21c7-4653-ba57-46a3be01d996","Type":"ContainerStarted","Data":"741f7d3f9e413977b74787d4dd72d1a0140c4c1779e93b5304a27a11ac7fd51e"}
Nov 02 21:26:40.437904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:40.437827    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podStartSLOduration=2.437811538 podStartE2EDuration="2.437811538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:26:40.437215328 +0000 UTC m=+829.981007242" watchObservedRunningTime="2026-11-02 21:26:40.437811538 +0000 UTC m=+829.981603491"
Nov 02 21:26:41.270538 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:41.270498    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:49.270441 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:49.270402    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:49.282934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:49.282880    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:49.392111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:26:49.392081    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:26:52.259161 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:26:52.259119    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:27:20.915255 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:20.915220    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 21:27:20.919325 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:20.919304    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:20.972618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:20.972585    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 21:27:21.017530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.017485    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-c2dd3b\"/\"openshift-service-ca.crt\""
Nov 02 21:27:21.017711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.017551    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-c2dd3b\"/\"kube-root-ca.crt\""
Nov 02 21:27:21.017711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.017609    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-c2dd3b\"/\"default-dockercfg-lnkl5\""
Nov 02 21:27:21.018952 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.018935    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-c2dd3b\"/\"premium-w1-c2dd3b-kserve-self-signed-certs\""
Nov 02 21:27:21.056314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056278    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.056494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056334    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.056494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056481    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.056577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056517    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stsps\" (UniqueName: \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.056630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056616    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.056696 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.056682    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157444    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157492    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157543    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157578    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157610    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stsps\" (UniqueName: \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.157691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157668    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.158008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157982    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.158070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.157997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.158123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.158102    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.159973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.159951    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.160404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.160382    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.212416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.212337    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stsps\" (UniqueName: \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\") pod \"premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") " pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.232742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.232712    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:21.604698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:21.604673    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 21:27:21.606640 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:27:21.606605    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b27adf3_f506_4702_92cd_becfd90326a3.slice/crio-07eb349896daf5c91e15a83cdd62b139af7edcb59856903fe9d00c6104f21503 WatchSource:0}: Error finding container 07eb349896daf5c91e15a83cdd62b139af7edcb59856903fe9d00c6104f21503: Status 404 returned error can't find the container with id 07eb349896daf5c91e15a83cdd62b139af7edcb59856903fe9d00c6104f21503
Nov 02 21:27:22.261182 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:27:22.261071    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:27:22.547398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:22.547308    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" event={"ID":"9b27adf3-f506-4702-92cd-becfd90326a3","Type":"ContainerStarted","Data":"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"}
Nov 02 21:27:22.547398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:22.547348    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" event={"ID":"9b27adf3-f506-4702-92cd-becfd90326a3","Type":"ContainerStarted","Data":"07eb349896daf5c91e15a83cdd62b139af7edcb59856903fe9d00c6104f21503"}
Nov 02 21:27:22.658204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:22.658153    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podStartSLOduration=2.658138328 podStartE2EDuration="2.658138328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:27:22.657312606 +0000 UTC m=+872.201104523" watchObservedRunningTime="2026-11-02 21:27:22.658138328 +0000 UTC m=+872.201930243"
Nov 02 21:27:23.233391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:23.233352    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:31.233395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:31.233360    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:31.246141 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:31.246107    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:31.604262 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:31.604180    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:27:41.455363 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.455323    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 21:27:41.458990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.458964    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.534896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.534835    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4e2b27\"/\"openshift-service-ca.crt\""
Nov 02 21:27:41.536712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.536686    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 21:27:41.556916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.556886    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4e2b27\"/\"kube-root-ca.crt\""
Nov 02 21:27:41.556916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.556901    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4e2b27\"/\"default-dockercfg-44bw2\""
Nov 02 21:27:41.557138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.556946    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4e2b27\"/\"premium-w4-4e2b27-kserve-self-signed-certs\""
Nov 02 21:27:41.569459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569430    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.569605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569462    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.569605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569559    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.569703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569655    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.569754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569733    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqkn\" (UniqueName: \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.569791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.569768    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670710    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670762    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqkn\" (UniqueName: \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670784    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670826    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670863    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.670963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.670934    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.671240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.671205    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.671298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.671276    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.671370 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.671345    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.673481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.673455    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.673634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.673615    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.730237 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.730152    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqkn\" (UniqueName: \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\") pod \"premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") " pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:41.771512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:41.771468    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:42.158575 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:42.158547    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 21:27:42.160161 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:27:42.160131    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefa18a42_9299_4636_ba86_0e06c7d3686e.slice/crio-c015afb2ffe9cbf5c931861a7510f42635695b2e73e6861f06cd95b722eb1d41 WatchSource:0}: Error finding container c015afb2ffe9cbf5c931861a7510f42635695b2e73e6861f06cd95b722eb1d41: Status 404 returned error can't find the container with id c015afb2ffe9cbf5c931861a7510f42635695b2e73e6861f06cd95b722eb1d41
Nov 02 21:27:42.657389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:42.657355    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" event={"ID":"efa18a42-9299-4636-ba86-0e06c7d3686e","Type":"ContainerStarted","Data":"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"}
Nov 02 21:27:42.657795 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:42.657403    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" event={"ID":"efa18a42-9299-4636-ba86-0e06c7d3686e","Type":"ContainerStarted","Data":"c015afb2ffe9cbf5c931861a7510f42635695b2e73e6861f06cd95b722eb1d41"}
Nov 02 21:27:42.767339 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:42.767283    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podStartSLOduration=1.7672695489999999 podStartE2EDuration="1.767269549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:27:42.766605166 +0000 UTC m=+892.310397083" watchObservedRunningTime="2026-11-02 21:27:42.767269549 +0000 UTC m=+892.311061465"
Nov 02 21:27:42.772096 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:42.772071    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:50.686253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:50.686218    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:27:50.689892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:50.689841    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:27:51.772537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:51.772497    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:51.784884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:51.784835    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:27:52.259637 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:27:52.259600    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:27:52.722324 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:27:52.722294    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:28:11.210071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.210033    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 21:28:11.213896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.213874    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.264955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.264909    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.265160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.264979    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.265160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.265088    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.265160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.265125    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.265340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.265168    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kspdf\" (UniqueName: \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.265340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.265219    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.301070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.301038    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9a6cf2\"/\"premium-w3-9a6cf2-kserve-self-signed-certs\""
Nov 02 21:28:11.314773 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.314734    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 21:28:11.373489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.373453    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.373770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.373739    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.373938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.373784    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.373938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.373843    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kspdf\" (UniqueName: \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.374118 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.374096    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.374449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.374415    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.375121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.375094    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.376258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.376028    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.378068 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.377109    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.378923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.378897    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.379481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.379455    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.429520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.429482    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kspdf\" (UniqueName: \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\") pod \"premium-w3-9a6cf2-kserve-5464db59d4-54pjs\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") " pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.525801 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.525720    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:11.898190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:11.898159    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 21:28:11.899999 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:28:11.899968    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod389b1624_08e6_4dc3_bff7_397abbea06f7.slice/crio-ffe74e5e4c6a12f4c5e5d86ace1d59054adaf6b8e581e336e389742089bedef2 WatchSource:0}: Error finding container ffe74e5e4c6a12f4c5e5d86ace1d59054adaf6b8e581e336e389742089bedef2: Status 404 returned error can't find the container with id ffe74e5e4c6a12f4c5e5d86ace1d59054adaf6b8e581e336e389742089bedef2
Nov 02 21:28:12.813448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:12.813417    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" event={"ID":"389b1624-08e6-4dc3-bff7-397abbea06f7","Type":"ContainerStarted","Data":"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"}
Nov 02 21:28:12.813448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:12.813454    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" event={"ID":"389b1624-08e6-4dc3-bff7-397abbea06f7","Type":"ContainerStarted","Data":"ffe74e5e4c6a12f4c5e5d86ace1d59054adaf6b8e581e336e389742089bedef2"}
Nov 02 21:28:12.924173 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:12.924119    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podStartSLOduration=1.924102275 podStartE2EDuration="1.924102275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:28:12.922248349 +0000 UTC m=+922.466040278" watchObservedRunningTime="2026-11-02 21:28:12.924102275 +0000 UTC m=+922.467894190"
Nov 02 21:28:13.526437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:13.526392    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:21.526922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:21.526885    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:21.539843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:21.539813    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:21.873738 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:28:21.873666    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:28:22.261357 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:28:22.261320    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:28:52.260620 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:28:52.260577    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:29:22.259457 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:29:22.259402    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:29:48.730092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.730041    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:29:48.733859 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.733826    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.785774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.785740    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:29:48.831075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.831031    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dd6574\"/\"openshift-service-ca.crt\""
Nov 02 21:29:48.831263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.831110    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dd6574\"/\"default-dockercfg-sm622\""
Nov 02 21:29:48.832057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.832037    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dd6574\"/\"distinct-w0-dd6574-kserve-self-signed-certs\""
Nov 02 21:29:48.832289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.832271    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dd6574\"/\"kube-root-ca.crt\""
Nov 02 21:29:48.879090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879055    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.879287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879100    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl5tm\" (UniqueName: \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.879287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879178    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.879390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879288    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.879390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879348    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.879390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.879373    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980554    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980606    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl5tm\" (UniqueName: \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980636    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980685    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980733    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.980962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.980771    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.981268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.981240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.981404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.981304    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.981404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.981334    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.982997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.982976    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:48.983268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:48.983250    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:49.035806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:49.035768    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl5tm\" (UniqueName: \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\") pod \"distinct-w0-dd6574-kserve-77b8668c5-5w9qb\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") " pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:49.045553 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:49.045408    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:49.443336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:49.443311    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:29:50.318733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:50.318696    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" event={"ID":"0ba31a8d-ca21-48d1-b13f-236423e3015b","Type":"ContainerStarted","Data":"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"}
Nov 02 21:29:50.319269 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:50.318746    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" event={"ID":"0ba31a8d-ca21-48d1-b13f-236423e3015b","Type":"ContainerStarted","Data":"e9897d6d337fa6c32a868c9ed9597721732e9abca7975dd3a492c908508625f8"}
Nov 02 21:29:50.428663 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:50.428595    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podStartSLOduration=2.428580622 podStartE2EDuration="2.428580622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:29:50.426296612 +0000 UTC m=+1019.970088527" watchObservedRunningTime="2026-11-02 21:29:50.428580622 +0000 UTC m=+1019.972372537"
Nov 02 21:29:51.046232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:51.046183    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:52.259797 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:29:52.259749    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:29:59.045687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:59.045626    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:59.059459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:59.059430    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:29:59.386168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:29:59.386091    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:30:00.177268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.177230    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:00.181156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.181135    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.200093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.200055    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptr8q\" (UniqueName: \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.200274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.200219    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.230742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.230712    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j922b\""
Nov 02 21:30:00.281456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.281416    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:00.302052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.302014    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.302314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.302294    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptr8q\" (UniqueName: \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.303318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.303292    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.356688 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.356654    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptr8q\" (UniqueName: \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.494192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.494114    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:00.860897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.860864    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:00.862640 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:30:00.862608    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf681f8f0_80a6_42d3_8254_b8d3257e7860.slice/crio-ada3a3e9cc2b760840321899be663add17042c99bf4a1c0d0c0522d62183a283 WatchSource:0}: Error finding container ada3a3e9cc2b760840321899be663add17042c99bf4a1c0d0c0522d62183a283: Status 404 returned error can't find the container with id ada3a3e9cc2b760840321899be663add17042c99bf4a1c0d0c0522d62183a283
Nov 02 21:30:00.865282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:00.865261    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:30:01.387480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:01.387446    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerStarted","Data":"ada3a3e9cc2b760840321899be663add17042c99bf4a1c0d0c0522d62183a283"}
Nov 02 21:30:08.428827 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:08.428794    4888 generic.go:356] "Generic (PLEG): container finished" podID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerID="89f9cca07f90fdf5b6aab48751cc5a699f2b0ba430e4b2bdd6dbc6cd206fee73" exitCode=60
Nov 02 21:30:08.429251 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:08.428880    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerDied","Data":"89f9cca07f90fdf5b6aab48751cc5a699f2b0ba430e4b2bdd6dbc6cd206fee73"}
Nov 02 21:30:08.429251 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:08.429140    4888 scope.go:122] "RemoveContainer" containerID="89f9cca07f90fdf5b6aab48751cc5a699f2b0ba430e4b2bdd6dbc6cd206fee73"
Nov 02 21:30:09.436083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:09.436048    4888 generic.go:356] "Generic (PLEG): container finished" podID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerID="1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3" exitCode=60
Nov 02 21:30:09.436551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:09.436130    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerDied","Data":"1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3"}
Nov 02 21:30:09.436551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:09.436187    4888 scope.go:122] "RemoveContainer" containerID="89f9cca07f90fdf5b6aab48751cc5a699f2b0ba430e4b2bdd6dbc6cd206fee73"
Nov 02 21:30:09.436551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:09.436460    4888 scope.go:122] "RemoveContainer" containerID="1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3"
Nov 02 21:30:09.436724 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:30:09.436702    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn_odh-ai-gateway-infra(f681f8f0-80a6-42d3-8254-b8d3257e7860)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860"
Nov 02 21:30:10.443035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:10.443005    4888 scope.go:122] "RemoveContainer" containerID="1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3"
Nov 02 21:30:10.443422 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:30:10.443232    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn_odh-ai-gateway-infra(f681f8f0-80a6-42d3-8254-b8d3257e7860)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860"
Nov 02 21:30:15.529146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.529113    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:30:15.534387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.534358    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.584332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.584298    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-9a6cf2-openshift-default-dockercfg-d2zhq\""
Nov 02 21:30:15.634164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.634126    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:30:15.667384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667345    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667398    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667419    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667528    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667592    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667670    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7gx9\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667758    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.667818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.667805    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768423    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768470    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7gx9\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768541    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768572    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768612    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768663    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768695    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768750    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.768999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.768791    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.769438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.769409    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.769787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.769761    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.770325 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.770302    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.771053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.771032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.771400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.771383    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.772860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.772819    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.773050 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.773031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.822935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.822829    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7gx9\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.823083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.823030    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:15.847207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:15.847171    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:16.261703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.261662    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:30:16.267370 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:30:16.267341    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347e20ea_7142_41bf_b0dd_30353da4c110.slice/crio-ace6f33d27ccb695573fabfc9560ace1a1745bf497b49d347bf1a66f1089905d WatchSource:0}: Error finding container ace6f33d27ccb695573fabfc9560ace1a1745bf497b49d347bf1a66f1089905d: Status 404 returned error can't find the container with id ace6f33d27ccb695573fabfc9560ace1a1745bf497b49d347bf1a66f1089905d
Nov 02 21:30:16.269871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.269817    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:16.269986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.269930    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:16.269986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.269980    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:16.483249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.483210    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" event={"ID":"347e20ea-7142-41bf-b0dd-30353da4c110","Type":"ContainerStarted","Data":"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"}
Nov 02 21:30:16.483420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.483279    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" event={"ID":"347e20ea-7142-41bf-b0dd-30353da4c110","Type":"ContainerStarted","Data":"ace6f33d27ccb695573fabfc9560ace1a1745bf497b49d347bf1a66f1089905d"}
Nov 02 21:30:16.595400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.595294    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podStartSLOduration=1.595279192 podStartE2EDuration="1.595279192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:30:16.593233669 +0000 UTC m=+1046.137025586" watchObservedRunningTime="2026-11-02 21:30:16.595279192 +0000 UTC m=+1046.139071108"
Nov 02 21:30:16.848990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:16.848901    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:17.848829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:17.848794    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:17.849254 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:17.848863    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:18.848471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:18.848437    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:18.848624 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:18.848494    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:19.848580 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:19.848541    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:19.849057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:19.848602    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:20.847899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:20.847830    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:20.848114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:20.847921    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:21.593600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:21.593566    4888 scope.go:122] "RemoveContainer" containerID="1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3"
Nov 02 21:30:21.847860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:21.847784    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:21.847860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:21.847837    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:22.267353 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:30:22.267291    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:30:22.518003 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:22.517911    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerStarted","Data":"878510708f371e5b1c2f0de256f7fa229a84c5656767dd2ce74d73f716ee491d"}
Nov 02 21:30:22.628182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:22.628076    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" podStartSLOduration=16.245274614 podStartE2EDuration="22.62805609s" totalImagesPullingTime="6.382781476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:30:22.625241172 +0000 UTC m=+1052.169033090" watchObservedRunningTime="2026-11-02 21:30:22.62805609 +0000 UTC m=+1052.171848007"
Nov 02 21:30:22.848352 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:22.848270    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:30:22.848352 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:22.848326    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 02 21:30:23.525841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.525805    4888 generic.go:356] "Generic (PLEG): container finished" podID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerID="878510708f371e5b1c2f0de256f7fa229a84c5656767dd2ce74d73f716ee491d" exitCode=60
Nov 02 21:30:23.526045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.525883    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerDied","Data":"878510708f371e5b1c2f0de256f7fa229a84c5656767dd2ce74d73f716ee491d"}
Nov 02 21:30:23.526045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.525951    4888 scope.go:122] "RemoveContainer" containerID="1ac58931e01fd0685f8cbcad675c54edae06eca4f220b3cfa4437bb559b2b4e3"
Nov 02 21:30:23.526332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.526307    4888 scope.go:122] "RemoveContainer" containerID="878510708f371e5b1c2f0de256f7fa229a84c5656767dd2ce74d73f716ee491d"
Nov 02 21:30:23.526627 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:30:23.526598    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn_odh-ai-gateway-infra(f681f8f0-80a6-42d3-8254-b8d3257e7860)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860"
Nov 02 21:30:23.642633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.642601    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:23.852345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.852262    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:23.852677 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.852650    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:23.853350 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:23.853331    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:30:24.682136 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.682104    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:24.794882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.794777    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\") pod \"f681f8f0-80a6-42d3-8254-b8d3257e7860\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") "
Nov 02 21:30:24.794882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.794876    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\" (UniqueName: \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\") pod \"f681f8f0-80a6-42d3-8254-b8d3257e7860\" (UID: \"f681f8f0-80a6-42d3-8254-b8d3257e7860\") "
Nov 02 21:30:24.795173 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.795147    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca" pod "f681f8f0-80a6-42d3-8254-b8d3257e7860" (UID: "f681f8f0-80a6-42d3-8254-b8d3257e7860"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:30:24.795275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.795262    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f681f8f0-80a6-42d3-8254-b8d3257e7860-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:30:24.797204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.797160    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q" pod "f681f8f0-80a6-42d3-8254-b8d3257e7860" (UID: "f681f8f0-80a6-42d3-8254-b8d3257e7860"). InnerVolumeSpecName "kube-api-access-ptr8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:30:24.895942 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:24.895903    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptr8q\" (UniqueName: \"kubernetes.io/projected/f681f8f0-80a6-42d3-8254-b8d3257e7860-kube-api-access-ptr8q\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:30:25.539920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:25.539883    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn" event={"ID":"f681f8f0-80a6-42d3-8254-b8d3257e7860","Type":"ContainerDied","Data":"ada3a3e9cc2b760840321899be663add17042c99bf4a1c0d0c0522d62183a283"}
Nov 02 21:30:25.539920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:25.539904    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"
Nov 02 21:30:25.540214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:25.539951    4888 scope.go:122] "RemoveContainer" containerID="878510708f371e5b1c2f0de256f7fa229a84c5656767dd2ce74d73f716ee491d"
Nov 02 21:30:25.694474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:25.694435    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:25.742035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:25.742003    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894250-tqwdn"]
Nov 02 21:30:26.601200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:26.601164    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" path="/var/lib/kubelet/pods/f681f8f0-80a6-42d3-8254-b8d3257e7860/volumes"
Nov 02 21:30:47.207537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207501    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207936    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207949    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207964    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207969    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207981    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.207986    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.208054    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.208062    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.208231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.208070    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f681f8f0-80a6-42d3-8254-b8d3257e7860" containerName="cleanup"
Nov 02 21:30:47.215602 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.215578    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.268196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.268157    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-dd6574-openshift-default-dockercfg-wcc2w\""
Nov 02 21:30:47.318428 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.318376    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:30:47.330461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330427    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330495    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330520    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvkv\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330539    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330596    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330624    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330650    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.330822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.330666    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431381 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431343    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431386    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvkv\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431519    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431559    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431611    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431646    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431682    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431707    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.431787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.431761    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.432047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.432023    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.432116 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.432030    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.432161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.432124    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.432223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.432184    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.432472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.432449    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.433946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.433923    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.434055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.433961    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.485923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.485837    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.486081 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.486058    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvkv\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.531464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.531425    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:47.946798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.946767    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:30:47.948111 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:30:47.948082    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42953e8_b649_4b08_8dda_c8439f0ac2a5.slice/crio-b543a8f5c420e575818bd31f780552387c2cf5bdf4af8397728d64fcbc3eb536 WatchSource:0}: Error finding container b543a8f5c420e575818bd31f780552387c2cf5bdf4af8397728d64fcbc3eb536: Status 404 returned error can't find the container with id b543a8f5c420e575818bd31f780552387c2cf5bdf4af8397728d64fcbc3eb536
Nov 02 21:30:47.950438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.950407    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:47.950526 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.950473    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:47.950526 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:47.950503    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:30:48.666823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:48.666786    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" event={"ID":"a42953e8-b649-4b08-8dda-c8439f0ac2a5","Type":"ContainerStarted","Data":"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"}
Nov 02 21:30:48.667246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:48.666841    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" event={"ID":"a42953e8-b649-4b08-8dda-c8439f0ac2a5","Type":"ContainerStarted","Data":"b543a8f5c420e575818bd31f780552387c2cf5bdf4af8397728d64fcbc3eb536"}
Nov 02 21:30:48.891520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:48.891460    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podStartSLOduration=1.891445013 podStartE2EDuration="1.891445013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:30:48.842199737 +0000 UTC m=+1078.385991652" watchObservedRunningTime="2026-11-02 21:30:48.891445013 +0000 UTC m=+1078.435236921"
Nov 02 21:30:49.534973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:49.534923    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:49.536067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:49.536049    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:49.536163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:49.536097    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:50.535080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:50.535043    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:50.535621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:50.535104    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:51.534238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:51.534198    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:51.534548 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:51.534520    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:52.275801 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:30:52.275762    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:30:52.533767 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:52.533243    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:52.533767 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:52.533307    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:53.535034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:53.534984    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:53.535594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:53.535074    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:54.532065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:54.532020    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:30:54.532262 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:54.532093    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 21:30:55.536440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:55.536408    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:55.536887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:55.536696    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:30:55.537571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:30:55.537551    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:31:10.298008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.297974    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 21:31:10.301686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.301658    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.354006 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.353970    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 21:31:10.398905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.398834    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4e2b27\"/\"distinct-w4-4e2b27-kserve-self-signed-certs\""
Nov 02 21:31:10.501319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501285    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.501481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501428    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.501559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501487    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.501559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501515    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.501635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501611    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.501669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.501660    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm8vc\" (UniqueName: \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602449    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602504    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm8vc\" (UniqueName: \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602552    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602642    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602703    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.602784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.602734    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.603163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.603142    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.603364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.603339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.603429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.603370    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.605143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.605124    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.605690 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.605671    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.656827 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.656790    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm8vc\" (UniqueName: \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\") pod \"distinct-w4-4e2b27-kserve-69df578557-rvzkt\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") " pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:10.913893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:10.913833    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:11.286880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:11.286832    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 21:31:11.287954 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:31:11.287922    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod277619f9_50d0_41c8_a4cf_2630a53de69a.slice/crio-27109b72f648a6c35e2544be53c900ee4923df1abf3a9f4b2440e64b55dd7999 WatchSource:0}: Error finding container 27109b72f648a6c35e2544be53c900ee4923df1abf3a9f4b2440e64b55dd7999: Status 404 returned error can't find the container with id 27109b72f648a6c35e2544be53c900ee4923df1abf3a9f4b2440e64b55dd7999
Nov 02 21:31:11.806146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:11.806113    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" event={"ID":"277619f9-50d0-41c8-a4cf-2630a53de69a","Type":"ContainerStarted","Data":"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"}
Nov 02 21:31:11.806564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:11.806154    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" event={"ID":"277619f9-50d0-41c8-a4cf-2630a53de69a","Type":"ContainerStarted","Data":"27109b72f648a6c35e2544be53c900ee4923df1abf3a9f4b2440e64b55dd7999"}
Nov 02 21:31:11.914970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:11.914930    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:11.919165 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:11.919117    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podStartSLOduration=1.919103021 podStartE2EDuration="1.919103021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:31:11.917327081 +0000 UTC m=+1101.461118998" watchObservedRunningTime="2026-11-02 21:31:11.919103021 +0000 UTC m=+1101.462894936"
Nov 02 21:31:20.914982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:20.914939    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:20.928585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:20.928553    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:21.874560 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:21.874523    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:31:22.259722 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:31:22.259681    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:31:45.935515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:45.935480    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:31:45.939530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:45.939505    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:45.989167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:45.989131    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-c2dd3b-openshift-default-dockercfg-rvfbh\""
Nov 02 21:31:46.040469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.040429    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:31:46.090652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.090606    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.090842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.090666    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlkw4\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.090842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.090825    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.090947    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.090987    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.091036    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.091127    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.091203    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.091282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.091229    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.192902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.192778    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.192902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.192820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.192902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.192843    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.192911    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlkw4\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.192985    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193023    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193056    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193080    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193129    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193226    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193318    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193425    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193498    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.193844 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.193824    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.195217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.195186    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.195460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.195442    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.248688 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.248642    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.248862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.248832    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlkw4\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.254687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.254662    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:46.459985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.459874    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:31:46.464074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.464044    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.515667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.515630    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-4d9ea0-openshift-default-dockercfg-vfpmv\""
Nov 02 21:31:46.518735 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.518706    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:31:46.598474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598433    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598499    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598632    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598771 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598728    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598771    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598873    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.598980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.598957    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.599037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.599015    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.599173 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.599097    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h74jp\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.671654 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.671618    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:31:46.672531 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:31:46.672505    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024f1b76_1c00_4b3f_94da_c38165e6c8f8.slice/crio-eee8bcdfc3513cff94c3adb29bf012f5afbec106b32b47baa59da0b6a3bfb136 WatchSource:0}: Error finding container eee8bcdfc3513cff94c3adb29bf012f5afbec106b32b47baa59da0b6a3bfb136: Status 404 returned error can't find the container with id eee8bcdfc3513cff94c3adb29bf012f5afbec106b32b47baa59da0b6a3bfb136
Nov 02 21:31:46.674987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.674951    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:46.675088 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.675022    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:46.675088 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.675055    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:46.700560 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700522    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700560 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700572    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700685    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700858 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700858 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700815    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74jp\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.700956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700914    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700964    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700993    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.700996    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.701320    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.701436    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.701519    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.701799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.701777    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.703060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.703037    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.703634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.703614    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.755284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.755254    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.755459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.755435    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h74jp\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:46.778250 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:46.778208    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:47.003071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.002983    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" event={"ID":"024f1b76-1c00-4b3f-94da-c38165e6c8f8","Type":"ContainerStarted","Data":"cfb2f7f99d9fc9e05fad9873045aaec440a58026664b2dbcc6f80d1552a71595"}
Nov 02 21:31:47.003071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.003033    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" event={"ID":"024f1b76-1c00-4b3f-94da-c38165e6c8f8","Type":"ContainerStarted","Data":"eee8bcdfc3513cff94c3adb29bf012f5afbec106b32b47baa59da0b6a3bfb136"}
Nov 02 21:31:47.116025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.115948    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podStartSLOduration=2.115928653 podStartE2EDuration="2.115928653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:31:47.112970941 +0000 UTC m=+1136.656762858" watchObservedRunningTime="2026-11-02 21:31:47.115928653 +0000 UTC m=+1136.659720567"
Nov 02 21:31:47.195183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.195156    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:31:47.197134 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:31:47.197076    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08e3782_a1f6_4ba7_b31c_d90a8560d7e3.slice/crio-58b54b80b2ccb003ef648eb2bf50c4d57cec443fec74efeff259f342b9af1453 WatchSource:0}: Error finding container 58b54b80b2ccb003ef648eb2bf50c4d57cec443fec74efeff259f342b9af1453: Status 404 returned error can't find the container with id 58b54b80b2ccb003ef648eb2bf50c4d57cec443fec74efeff259f342b9af1453
Nov 02 21:31:47.199781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.199750    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:47.199880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.199818    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:47.199880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.199859    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:31:47.255434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.255341    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:47.260872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.256830    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:47.260872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:47.256925    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:48.022881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.022597    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" event={"ID":"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3","Type":"ContainerStarted","Data":"a68c6fb6ca3880430a70dc7fdac11f532da5172f5eea968f416877a3a5295893"}
Nov 02 21:31:48.023411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.022891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" event={"ID":"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3","Type":"ContainerStarted","Data":"58b54b80b2ccb003ef648eb2bf50c4d57cec443fec74efeff259f342b9af1453"}
Nov 02 21:31:48.146878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.146311    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podStartSLOduration=2.1462891 podStartE2EDuration="2.1462891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:31:48.144220261 +0000 UTC m=+1137.688012178" watchObservedRunningTime="2026-11-02 21:31:48.1462891 +0000 UTC m=+1137.690081017"
Nov 02 21:31:48.256875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.256368    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:48.256875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.256437    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:48.779025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.778985    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:48.780605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.780578    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:48.780743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:48.780634    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:49.257872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:49.257265    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:49.257872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:49.257328    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:49.778671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:49.778639    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:49.778866 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:49.778689    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:50.260695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:50.260267    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:50.260695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:50.260330    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:50.778727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:50.778694    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:50.778950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:50.778744    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:51.256316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:51.256275    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:51.256671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:51.256636    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:51.779415 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:51.779374    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:51.779918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:51.779440    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:52.262297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:52.262254    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:52.262495 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:52.262321    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:52.287990 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:31:52.287957    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:31:52.778982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:52.778944    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:52.779163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:52.779001    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:53.255861 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:53.255817    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:31:53.256246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:53.255901    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 02 21:31:53.778696 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:53.778647    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:31:53.778951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:53.778720    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 21:31:54.260129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.260099    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:54.260570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.260379    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:54.261188 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.261166    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:31:54.782927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.782893    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:54.783226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.783201    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:31:54.783993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:31:54.783976    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:32:01.887409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:01.887362    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:32:01.892257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:01.892229    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:01.941945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:01.941908    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-ef6b43-openshift-default-dockercfg-2pk6b\""
Nov 02 21:32:01.992261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:01.992231    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:32:02.018405 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018368    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018436    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018513    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4z4h\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018689    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018741    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018896    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.018977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.018937    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.119904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.119842    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.119919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.119946    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.119980    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120001    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z4h\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120054    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120100    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120125    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120151    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120396    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120448    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120486    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120524    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.120941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.120919    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.122516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.122486    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.122831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.122811    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.178695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.178658    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4z4h\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.179532 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.179507    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.186948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.186916    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:32:02.192182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.192161    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.205238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.205215    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:02.242570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.242527    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-4e2b27-openshift-default-dockercfg-knh48\""
Nov 02 21:32:02.247040 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.246961    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:32:02.322340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322301    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322409    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322505    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322653 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322568    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322652    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322723    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6ss\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322757    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.322971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.322907    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424521    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424604    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424623    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424708    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424764    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6ss\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424806    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424865    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.424947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424896    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.424975    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.425065    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.425267    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.425327    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.425415    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.425726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.425700    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.427338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.427308    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.427493 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.427475    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.479348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.479281    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.479491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.479471    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6ss\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.509530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.509493    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:02.624740 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.624709    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:32:02.627057 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:32:02.626991    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7957c066_92a0_48d7_944f_663a277e87f9.slice/crio-ce2dc6cefbd37d3f525fdae9d562e8cc05baea28b64a92671dcf40bf87d617be WatchSource:0}: Error finding container ce2dc6cefbd37d3f525fdae9d562e8cc05baea28b64a92671dcf40bf87d617be: Status 404 returned error can't find the container with id ce2dc6cefbd37d3f525fdae9d562e8cc05baea28b64a92671dcf40bf87d617be
Nov 02 21:32:02.628810 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.628779    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:02.628934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.628864    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:02.628934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.628903    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:02.879369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.879336    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:32:02.880174 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:32:02.880146    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5e927f_53ae_4334_b4a3_0b5935d362ae.slice/crio-9f9565d3d9c86af345de079f065009d32ba868f9984783ca528bb3911479a193 WatchSource:0}: Error finding container 9f9565d3d9c86af345de079f065009d32ba868f9984783ca528bb3911479a193: Status 404 returned error can't find the container with id 9f9565d3d9c86af345de079f065009d32ba868f9984783ca528bb3911479a193
Nov 02 21:32:02.882821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.882790    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:02.882914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.882887    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:02.882951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:02.882931    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:03.158884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.158125    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" event={"ID":"7d5e927f-53ae-4334-b4a3-0b5935d362ae","Type":"ContainerStarted","Data":"a8d2615c232cfa2176bc832234ad30343cc9f0f62f7859409ed5b09ac769c5a7"}
Nov 02 21:32:03.158884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.158188    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" event={"ID":"7d5e927f-53ae-4334-b4a3-0b5935d362ae","Type":"ContainerStarted","Data":"9f9565d3d9c86af345de079f065009d32ba868f9984783ca528bb3911479a193"}
Nov 02 21:32:03.162226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.161527    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" event={"ID":"7957c066-92a0-48d7-944f-663a277e87f9","Type":"ContainerStarted","Data":"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"}
Nov 02 21:32:03.162226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.161580    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" event={"ID":"7957c066-92a0-48d7-944f-663a277e87f9","Type":"ContainerStarted","Data":"ce2dc6cefbd37d3f525fdae9d562e8cc05baea28b64a92671dcf40bf87d617be"}
Nov 02 21:32:03.205876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.205834    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:03.207679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.207654    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:03.207791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.207709    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:03.279599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.279535    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podStartSLOduration=1.279516464 podStartE2EDuration="1.279516464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:32:03.277007025 +0000 UTC m=+1152.820798941" watchObservedRunningTime="2026-11-02 21:32:03.279516464 +0000 UTC m=+1152.823308385"
Nov 02 21:32:03.392903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.392816    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podStartSLOduration=2.392798479 podStartE2EDuration="2.392798479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:32:03.390702916 +0000 UTC m=+1152.934494836" watchObservedRunningTime="2026-11-02 21:32:03.392798479 +0000 UTC m=+1152.936590396"
Nov 02 21:32:03.515549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.514249    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:03.534877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.521979    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:03.534877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:03.522043    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:04.206549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:04.206504    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:04.206960 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:04.206575    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:04.514406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:04.514309    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:04.514661 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:04.514632    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:05.205871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:05.205767    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:05.205871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:05.205820    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:05.521928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:05.518979    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:05.521928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:05.519044    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:06.206369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:06.206330    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:06.206530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:06.206395    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:06.514403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:06.514305    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:06.514679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:06.514649    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:07.206461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:07.206423    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:07.206892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:07.206484    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:07.519933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:07.514296    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:07.519933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:07.514359    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:08.205679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:08.205647    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:08.205956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:08.205933    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:08.513930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:08.510273    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:08.513930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:08.510355    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:09.206774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:09.206732    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:32:09.206980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:09.206803    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 02 21:32:09.510286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:09.510200    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:32:09.510286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:09.510269    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 02 21:32:10.210633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.210603    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:10.211096 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.211075    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:10.211898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.211866    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:32:10.514758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.514670    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:10.515012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.514991    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:10.515671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:10.515652    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:32:22.259835 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:32:22.259797    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:32:31.530790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.530754    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:32:31.534530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.534510    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641226    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641278    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641304    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641558    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641698    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641782 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641762    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641790    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641919 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641897    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhndl\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.641977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.641936    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.725138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.725101    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:32:31.743683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743649    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.743886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743738    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.743886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.743886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743799    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.743886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743835    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhndl\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743889    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.743984    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744050    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744271    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744287    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744376    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744643    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.744927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.744904    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.746924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.746901    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.747387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.747367    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.797905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.797813    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.798052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.797909    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhndl\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:31.848028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:31.847992    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:32.269859 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.269813    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:32:32.270843 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:32:32.270818    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5f5cdb_9306_4828_9f7c_b3efe1455c7c.slice/crio-546adf7b9600d22684b1436eac115157a5be38ac88ab9bce9019f79fdf60a1ea WatchSource:0}: Error finding container 546adf7b9600d22684b1436eac115157a5be38ac88ab9bce9019f79fdf60a1ea: Status 404 returned error can't find the container with id 546adf7b9600d22684b1436eac115157a5be38ac88ab9bce9019f79fdf60a1ea
Nov 02 21:32:32.273308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.273273    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:32.273407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.273361    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:32.273471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.273409    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:32.342496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.342459    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" event={"ID":"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c","Type":"ContainerStarted","Data":"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"}
Nov 02 21:32:32.342698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.342519    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" event={"ID":"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c","Type":"ContainerStarted","Data":"546adf7b9600d22684b1436eac115157a5be38ac88ab9bce9019f79fdf60a1ea"}
Nov 02 21:32:32.453374 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.453304    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podStartSLOduration=1.45328853 podStartE2EDuration="1.45328853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:32:32.45196389 +0000 UTC m=+1181.995755817" watchObservedRunningTime="2026-11-02 21:32:32.45328853 +0000 UTC m=+1181.997080445"
Nov 02 21:32:32.849985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:32.849941    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:33.026876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.021812    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:32:33.026876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.026654    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.059737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.059704    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.059920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.059761    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.059920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.059784    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.059985    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.060014    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.060037    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.060061    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.060177    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.060240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.060226    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7cmd\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.069077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.069046    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:32:33.161259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161219    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161259 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161332    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161349    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161373    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161407    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161437    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161476    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7cmd\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161564    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.161956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161904    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.162010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161994    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.162066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.161998    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.162236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.162211    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.162344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.162242    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.163913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.163892    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.164158 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.164141    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.237877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.237379    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.238203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.238177    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7cmd\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.354162 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.354124    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:33.787914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.787878    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:32:33.788571 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:32:33.788544    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba80d8e6_2176_4023_b009_e9da7e6c9c83.slice/crio-21e28e0225c520de70215af2bf1a792032bf9d8a3e1d2bff320ef6749838082d WatchSource:0}: Error finding container 21e28e0225c520de70215af2bf1a792032bf9d8a3e1d2bff320ef6749838082d: Status 404 returned error can't find the container with id 21e28e0225c520de70215af2bf1a792032bf9d8a3e1d2bff320ef6749838082d
Nov 02 21:32:33.791776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.791747    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:33.791876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.791836    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:33.791922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.791896    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:32:33.850750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.850718    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:33.851140 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:33.850770    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:34.355020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:34.354983    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" event={"ID":"ba80d8e6-2176-4023-b009-e9da7e6c9c83","Type":"ContainerStarted","Data":"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"}
Nov 02 21:32:34.355191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:34.355047    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" event={"ID":"ba80d8e6-2176-4023-b009-e9da7e6c9c83","Type":"ContainerStarted","Data":"21e28e0225c520de70215af2bf1a792032bf9d8a3e1d2bff320ef6749838082d"}
Nov 02 21:32:34.577232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:34.577160    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podStartSLOduration=2.577146988 podStartE2EDuration="2.577146988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:32:34.529449432 +0000 UTC m=+1184.073241354" watchObservedRunningTime="2026-11-02 21:32:34.577146988 +0000 UTC m=+1184.120938904"
Nov 02 21:32:34.859875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:34.850693    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:34.859875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:34.850760    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:35.359448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:35.359401    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:35.361541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:35.361505    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:35.361794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:35.361765    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:35.854878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:35.854396    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:35.854878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:35.854464    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:36.355499 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:36.355450    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:36.356155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:36.356107    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:36.850876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:36.850550    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:36.850876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:36.850653    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:37.357312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:37.357273    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:37.357966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:37.357928    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:37.855354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:37.855306    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:37.855580 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:37.855376    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:38.360312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:38.360277    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:38.360991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:38.360952    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:38.848423 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:38.848385    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 02 21:32:38.848613 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:38.848443    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 02 21:32:39.354881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:39.354814    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:39.355063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:39.354903    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:40.022590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:40.022355    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:40.023376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:40.023355    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:40.024305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:40.024268    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:32:40.355280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:40.355193    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 21:32:40.355280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:40.355246    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 21:32:41.358970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:41.358941    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:41.359444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:41.359238    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:41.359830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:41.359811    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:32:50.730511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:50.730479    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:32:50.732219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:32:50.732195    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:32:52.260551 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:32:52.260514    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:33:02.944623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:02.944529    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:33:02.950036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:02.950008    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.047109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.047072    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-284b8e-openshift-default-dockercfg-w2r92\""
Nov 02 21:33:03.050485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.050450    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:33:03.076911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.076839    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.076982    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077075    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077099    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077184    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077213    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077231    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077308    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.077378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.077355    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l4wd\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178567    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178642    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178664    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178686    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178715    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.178796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178753    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178803    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178899    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l4wd\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.178987    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179271 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.179107    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179405 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.179378    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.179421    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.179489    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.179842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.179823    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.181052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.181030    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.181428 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.181410    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.234025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.233951    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.234230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.234201    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l4wd\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.265272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.265242    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:03.680484 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.680458    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:33:03.681861 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:33:03.681823    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f23da92_5f6a_40ed_9ea1_bfc24b8e2d0d.slice/crio-2ae2b7763cba6dbc593ce54d9f14fb74c69b520cc946c74a35dd16e1a0cd6c25 WatchSource:0}: Error finding container 2ae2b7763cba6dbc593ce54d9f14fb74c69b520cc946c74a35dd16e1a0cd6c25: Status 404 returned error can't find the container with id 2ae2b7763cba6dbc593ce54d9f14fb74c69b520cc946c74a35dd16e1a0cd6c25
Nov 02 21:33:03.684510 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.684475    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:03.684616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.684546    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:03.684616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:03.684574    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:04.541831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:04.541793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" event={"ID":"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d","Type":"ContainerStarted","Data":"5fd06cb962cd286c4c741bf8452444b6e14d481eb9a568437cfb522872082024"}
Nov 02 21:33:04.542382 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:04.541879    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" event={"ID":"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d","Type":"ContainerStarted","Data":"2ae2b7763cba6dbc593ce54d9f14fb74c69b520cc946c74a35dd16e1a0cd6c25"}
Nov 02 21:33:04.702530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:04.702472    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podStartSLOduration=2.702459301 podStartE2EDuration="2.702459301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:04.700743919 +0000 UTC m=+1214.244535836" watchObservedRunningTime="2026-11-02 21:33:04.702459301 +0000 UTC m=+1214.246251217"
Nov 02 21:33:05.265768 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:05.265729    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:05.267357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:05.267329    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:05.267465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:05.267377    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:06.266064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:06.266022    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:06.266523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:06.266086    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:07.266179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:07.266134    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:07.266629 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:07.266193    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:08.266246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:08.266204    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:08.266676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:08.266262    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:09.266340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:09.266306    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:09.266743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:09.266364    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:10.266396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:10.266360    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:33:10.266822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:10.266427    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 21:33:11.270796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:11.270765    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:11.271325 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:11.271260    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:11.272159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:11.272140    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:33:22.262919 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:33:22.262882    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:33:48.578337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.578301    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:33:48.582469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.582444    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666301    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666399    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666429    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666524    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666581    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666623    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtb62\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666898    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.666997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.666953    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.667070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.667048    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.771905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.771862    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.772656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.772632    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773191    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773368    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773515    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773662    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.773918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773904    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.776467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.773139    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.776467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.774434    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.776467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.772547    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.776467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.775945    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.776810 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.776791    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtb62\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.777765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.777743    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.778984 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.778962    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:33:48.795268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.783439    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.813452 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.813413    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.830274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.830196    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.830730 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.830698    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtb62\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\") pod \"e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") " pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:48.899218 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:48.899181    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:49.318340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.318317    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:33:49.319315 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:33:49.319287    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac93a91_7851_42ea_b49e_6d4db406ee0c.slice/crio-6193ed9eb1b348b6e4e88c5b40038ea05fb0ac60b99feefad5c9ef93027901b3 WatchSource:0}: Error finding container 6193ed9eb1b348b6e4e88c5b40038ea05fb0ac60b99feefad5c9ef93027901b3: Status 404 returned error can't find the container with id 6193ed9eb1b348b6e4e88c5b40038ea05fb0ac60b99feefad5c9ef93027901b3
Nov 02 21:33:49.321953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.321919    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:49.322037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.321985    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:49.322037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.322015    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:33:49.897309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.897265    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" event={"ID":"8ac93a91-7851-42ea-b49e-6d4db406ee0c","Type":"ContainerStarted","Data":"dd43d8d26818e4014b8e60aa47a6616d4ba8ed36d8658b0adc269bad2fc3615d"}
Nov 02 21:33:49.897708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.897327    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" event={"ID":"8ac93a91-7851-42ea-b49e-6d4db406ee0c","Type":"ContainerStarted","Data":"6193ed9eb1b348b6e4e88c5b40038ea05fb0ac60b99feefad5c9ef93027901b3"}
Nov 02 21:33:49.899643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:49.899622    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:33:50.008168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:50.008115    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podStartSLOduration=2.008100784 podStartE2EDuration="2.008100784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:50.006911528 +0000 UTC m=+1259.550703447" watchObservedRunningTime="2026-11-02 21:33:50.008100784 +0000 UTC m=+1259.551892699"
Nov 02 21:33:50.902670 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:50.902630    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:50.903280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:50.903249    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:51.900506 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:51.900475    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:51.900668 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:51.900526    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:52.267015 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:33:52.266969    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:33:52.900598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:52.900558    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:52.900781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:52.900618    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:53.899826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:53.899788    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:53.900253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:53.899867    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:54.899905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:54.899836    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:54.900416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:54.899949    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:55.904804 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:55.904757    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:55.905444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:55.905409    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:56.912872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:56.912270    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:33:56.912872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:56.912354    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 21:33:57.597489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:57.597444    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:57.671023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:57.670972    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:57.734995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:57.734944    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:33:57.874878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:57.874742    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:58.135073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.134959    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:58.215067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.215005    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:58.307994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.307944    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:58.382012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.381939    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:33:58.383430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.383395    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:58.769870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.769510    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:33:58.794879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.770306    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:33:58.931599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.931560    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:33:58.931979 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:58.931947    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:33:59.059927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:59.056982    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:33:59.977825 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:59.977509    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:33:59.978942 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:33:59.977576    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:00.268220 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:00.268128    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:34:00.269307 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:00.269277    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:34:00.271964 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:00.271937    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:34:02.505776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.505738    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 21:34:02.510030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.509992    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.573962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.573919    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.573962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.573969    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.573987    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574102    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfkk8\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574265    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574285    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574434    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.574494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.574480    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.617144 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.617107    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 21:34:02.675916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.675874    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.675930    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.675954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676010    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676044    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676103    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676147    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676180    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfkk8\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676343    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676441    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.676669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.676631    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.677023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.677001    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.677226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.677207    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.678487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.678467    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.679192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.679175    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.732070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.732034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.732678 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.732656    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfkk8\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:02.823820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:02.823703    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:03.245712 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:03.245676    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf693530c_15d8_4dc1_af8f_69e54a31d6ee.slice/crio-ee4190dd5c6efa37168116f16246a92dd3f22ceee3b724fb0ae359ae13f188d9 WatchSource:0}: Error finding container ee4190dd5c6efa37168116f16246a92dd3f22ceee3b724fb0ae359ae13f188d9: Status 404 returned error can't find the container with id ee4190dd5c6efa37168116f16246a92dd3f22ceee3b724fb0ae359ae13f188d9
Nov 02 21:34:03.246272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:03.246168    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 21:34:03.248443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:03.248406    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:03.248558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:03.248471    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:03.248558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:03.248501    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:04.100420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.100381    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" event={"ID":"f693530c-15d8-4dc1-af8f-69e54a31d6ee","Type":"ContainerStarted","Data":"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"}
Nov 02 21:34:04.100920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.100441    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" event={"ID":"f693530c-15d8-4dc1-af8f-69e54a31d6ee","Type":"ContainerStarted","Data":"ee4190dd5c6efa37168116f16246a92dd3f22ceee3b724fb0ae359ae13f188d9"}
Nov 02 21:34:04.257614 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.257551    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podStartSLOduration=2.25753613 podStartE2EDuration="2.25753613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:04.209694841 +0000 UTC m=+1273.753486757" watchObservedRunningTime="2026-11-02 21:34:04.25753613 +0000 UTC m=+1273.801328046"
Nov 02 21:34:04.830376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.828313    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:04.830713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.830684    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:04.830915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:04.830885    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:05.834911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:05.832892    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:05.834911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:05.832953    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:06.829875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:06.826256    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:06.829875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:06.826332    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:07.830880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:07.830260    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:07.830880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:07.830326    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:08.833344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:08.833301    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:08.833973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:08.833940    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:09.824127 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:09.824083    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:34:09.824313 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:09.824157    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 21:34:10.829443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:10.829412    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:10.829898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:10.829876    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:10.830764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:10.830747    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 21:34:17.068200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.068162    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:34:17.072460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.072427    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.188790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.188747    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b6w8\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.188810    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.188945    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189024    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189079    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189106    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189224    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189290    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.189411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.189389    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290366    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290433    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290476    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b6w8\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290497    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290625    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290734    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290773    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290801    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290814    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.290932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290903    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.291076 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.290981    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.291463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.291428    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.291578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.291495    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.291621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.291601    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.293198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.293177    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.293400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.293382    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.300134 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.300108    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:34:17.345996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.345921    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.346149 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.346055    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b6w8\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.388883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.388842    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:17.650197 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.647722    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:34:17.655973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.655941    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.676974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.676938    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:34:17.799839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.799797    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.799927    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.799996    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800033    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800083    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800125    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800221    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800275    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsnz8\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.800363 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.800349    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.826806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.826775    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:34:17.828347 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:17.828322    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54a745f_4dd9_4b1d_ad14_580491ac1637.slice/crio-7894cc2193395fe0455b44306ffb6ade3751dfdbd18ad5a61c8461e893fbbf37 WatchSource:0}: Error finding container 7894cc2193395fe0455b44306ffb6ade3751dfdbd18ad5a61c8461e893fbbf37: Status 404 returned error can't find the container with id 7894cc2193395fe0455b44306ffb6ade3751dfdbd18ad5a61c8461e893fbbf37
Nov 02 21:34:17.830963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.830932    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:17.831045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.830998    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:17.831045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.831025    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:17.899312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.899277    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:34:17.901196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901121    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901167    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901227    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901345    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901411    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsnz8\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901465    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901508    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901557    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.901687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.901589    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.903036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.902996    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.903036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.903034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.903229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.903207    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.903583 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.903564    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.903721 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.903704    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.904827 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.904808    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:17.905198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.905179    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.974492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.973405    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsnz8\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:17.974492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.973728    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:34:17.975279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:17.975256    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\") pod \"e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") " pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:18.001438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.001405    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:18.002805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.002778    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.002945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.002838    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003110 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003084    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003131    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003266    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003340    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003425    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqn7b\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.003502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.003483    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.104755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104784    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104822    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104882    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqn7b\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104921    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.104975    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.105040    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.105173    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.105243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.105205    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.106054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.106029    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.106490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.106468    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.106888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.106842    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.107016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.106975    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.107315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.107291    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.109310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.109289    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.109823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.109800    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.161792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.161757    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqn7b\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.161971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.161756    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\") pod \"e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") " pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.218800 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.218765    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:18.226842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.226801    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" event={"ID":"e54a745f-4dd9-4b1d-ad14-580491ac1637","Type":"ContainerStarted","Data":"369f0f12f21374ea0ff2a90fdcae1243f1342e9483d786c64048eab2bc9cb909"}
Nov 02 21:34:18.227287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.226997    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" event={"ID":"e54a745f-4dd9-4b1d-ad14-580491ac1637","Type":"ContainerStarted","Data":"7894cc2193395fe0455b44306ffb6ade3751dfdbd18ad5a61c8461e893fbbf37"}
Nov 02 21:34:18.232739 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.232710    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:34:18.236727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.236706    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.297252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.297210    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308626    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308723    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308799    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308835    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308903    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308927    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.308957    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.309082    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwrc\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.315243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.309195    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.341967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.341485    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podStartSLOduration=1.341472228 podStartE2EDuration="1.341472228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:18.338595397 +0000 UTC m=+1287.882387319" watchObservedRunningTime="2026-11-02 21:34:18.341472228 +0000 UTC m=+1287.885264143"
Nov 02 21:34:18.378126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.378092    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:34:18.378375 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:18.378351    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3c7cb9_23a6_49a5_ac4f_e633f02e1bd5.slice/crio-cde785da491b1790fe73b085a37b985e51250f316a65d221df9ad81a62d6882b WatchSource:0}: Error finding container cde785da491b1790fe73b085a37b985e51250f316a65d221df9ad81a62d6882b: Status 404 returned error can't find the container with id cde785da491b1790fe73b085a37b985e51250f316a65d221df9ad81a62d6882b
Nov 02 21:34:18.381824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.381782    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.381959 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.381887    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.381959 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.381932    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.390578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.390272    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:18.391882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.391699    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:18.391882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.391751    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414202    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414277    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414318    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414374    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414399    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414436    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414548    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwrc\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414664    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.414716    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.415910    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.416247    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.417114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.416482    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.419351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.418997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.420064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.419650    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.421939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.421889    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.424014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.423969    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.474680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.474650    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.474807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.474786    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwrc\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.553108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.553073    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:18.609405 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.609365    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.609532 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.609457    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.609532 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.609503    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.620111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.617439    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:34:18.975007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.974974    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:34:18.976181 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:18.976142    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc80333_b572_4a7d_99b4_fdde6fc9ba7f.slice/crio-ae166d6c9a6c5bf0edce3ce07a929c5cfe32763d57e2fec13f783500d57259c0 WatchSource:0}: Error finding container ae166d6c9a6c5bf0edce3ce07a929c5cfe32763d57e2fec13f783500d57259c0: Status 404 returned error can't find the container with id ae166d6c9a6c5bf0edce3ce07a929c5cfe32763d57e2fec13f783500d57259c0
Nov 02 21:34:18.979159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.979127    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.979240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.979193    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:18.979240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:18.979221    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:19.299748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.299662    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" event={"ID":"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5","Type":"ContainerStarted","Data":"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"}
Nov 02 21:34:19.301468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.300356    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" event={"ID":"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5","Type":"ContainerStarted","Data":"cde785da491b1790fe73b085a37b985e51250f316a65d221df9ad81a62d6882b"}
Nov 02 21:34:19.304048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.304022    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" event={"ID":"f700027b-6535-4875-bab0-270c68fff872","Type":"ContainerStarted","Data":"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"}
Nov 02 21:34:19.304238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.304216    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" event={"ID":"f700027b-6535-4875-bab0-270c68fff872","Type":"ContainerStarted","Data":"863f134623430f50f96c8257a008c5e4335b518522c3a6b5cea88958938fc821"}
Nov 02 21:34:19.308942 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.308918    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" event={"ID":"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f","Type":"ContainerStarted","Data":"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"}
Nov 02 21:34:19.309108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.309090    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" event={"ID":"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f","Type":"ContainerStarted","Data":"ae166d6c9a6c5bf0edce3ce07a929c5cfe32763d57e2fec13f783500d57259c0"}
Nov 02 21:34:19.391522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.391488    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:19.391675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.391544    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:19.457871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.456829    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podStartSLOduration=2.456810797 podStartE2EDuration="2.456810797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:19.455699654 +0000 UTC m=+1288.999491571" watchObservedRunningTime="2026-11-02 21:34:19.456810797 +0000 UTC m=+1289.000602714"
Nov 02 21:34:19.555043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.554953    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:19.574907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:19.574811    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podStartSLOduration=2.574788251 podStartE2EDuration="2.574788251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:19.572336416 +0000 UTC m=+1289.116128337" watchObservedRunningTime="2026-11-02 21:34:19.574788251 +0000 UTC m=+1289.118580170"
Nov 02 21:34:20.007518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.007484    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:20.027872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.021029    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:20.027872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.021128    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:20.230182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.230127    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:20.230489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.230445    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:20.230998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.230980    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:20.399878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.399774    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:20.400432 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.400400    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:20.555897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.554829    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:20.555897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:20.554926    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:21.007344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.007302    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:21.007649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.007622    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:21.136892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.136138    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:21.225318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.225277    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:21.225599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.225568    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:21.226166 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.226140    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:21.341563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.341463    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:21.395730 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.395693    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:34:21.397604 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.397303    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:21.398265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.398235    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:21.558876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.558284    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:21.558876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.558345    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:21.613500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.613397    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:34:21.890078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.889969    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:21.890695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.890664    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:34:21.917000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:21.916941    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": context deadline exceeded"
Nov 02 21:34:22.005697 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.005661    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:22.006019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.005989    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:22.064988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.064940    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:34:22.216874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.194754    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:22.313084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.245568    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:22.315384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.245633    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:22.315736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.245683    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:22.328567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.314150    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:34:22.329731 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.329699    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:34:22.395964 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.370351    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:34:22.450769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.396287    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:34:22.450769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.397293    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:22.450769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.398250    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:34:22.450769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.398599    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:34:22.506950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.461568    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podStartSLOduration=4.461546365 podStartE2EDuration="4.461546365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:19.699242778 +0000 UTC m=+1289.243034698" watchObservedRunningTime="2026-11-02 21:34:22.461546365 +0000 UTC m=+1292.005338284"
Nov 02 21:34:22.558071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.558034    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:22.558336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.558309    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:22.562071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.562042    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:22.563318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.563284    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:22.577225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.315339    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:22.745032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.744988    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:22.936820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:22.936689    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": context deadline exceeded"
Nov 02 21:34:23.052862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.052807    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:34:23.114090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.114038    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:23.114430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.114398    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:23.118227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.118180    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:34:23.144171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.144112    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.260948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.256881    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:23.260948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.256967    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:23.321890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.320129    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.329890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.322412    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:34:23.394896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.388939    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.397679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.397635    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.399175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.399131    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.464921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.445201    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:23.464921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.445279    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:23.559129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.552207    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:34:23.559129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.552286    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:34:23.559129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.552322    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:34:23.559129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.552361    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:34:23.559129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.552396    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:34:23.583873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.579749    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:23.648872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.647922    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:23.773798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.773754    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:23.849931 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:34:23.849087    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:34:23.997873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:23.957980    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:34:24.029873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.007350    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:24.029873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.007409    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:24.230880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.230242    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:24.230880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.230308    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:24.230880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.230393    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:24.230880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.230665    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:34:24.318504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.318468    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:34:24.331035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.330987    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:24.331684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.331286    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:24.414334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.414281    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:34:24.414999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.414969    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:34:24.416280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.416257    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:24.416438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.416418    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:24.588352 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.588255    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:24.599349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.599306    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:24.660567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.660534    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:34:24.844932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.820026    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.003031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:24.963932    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.067584 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.067549    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:25.068450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.068412    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:25.228887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.221965    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:25.272951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.230645    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.496731 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.230748    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:25.497212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.284765    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:25.497371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.331929    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.497680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.331991    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:25.498047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.347894    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:25.498376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.389334    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:25.498533 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.498504    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:25.498637 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.416017    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.606279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.606211    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:25.625909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.613202    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:25.626227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.626171    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:25.804588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.804496    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:34:25.894989 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.894943    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:34:25.927535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:25.927507    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:34:26.042350 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.042312    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:26.042634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.042601    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:26.253170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.253137    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:34:26.314186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.314153    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:34:26.365064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.365024    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:26.365336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.365311    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:26.385658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.385346    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:34:26.391384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.391233    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:34:26.401195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.401159    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 21:34:26.401763 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.401721    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 21:34:26.554016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.553925    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:26.555711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:26.555679    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:27.002274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.002242    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 21:34:27.002764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.002724    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 21:34:27.222178 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.222137    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:34:27.222392 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.222210    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 02 21:34:27.401087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.401001    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:27.401446 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.401424    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:27.402220 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.402201    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:34:27.555212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.555174    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:34:27.555996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:27.555962    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:34:28.229875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.229820    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:28.231164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.231136    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:28.236128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.236102    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:34:28.346744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.345771    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:28.346744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.346572    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:28.353445 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.353380    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:34:28.608767 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.608657    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:28.633953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.633922    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:28.634275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:28.634254    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:34:32.452683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.452596    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:34:32.457999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.457975    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523302 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523266    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523327    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523483    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523637 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523533    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jdrs\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523632    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523780 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523759    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523814    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523884    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.523983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.523967    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.558004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.557974    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:34:32.625760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.625717    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.625997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.625877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.625997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.625920    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626010    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626116    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626235 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626210    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626274    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626319    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jdrs\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626359    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626488 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626391    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626660    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626801    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626866    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.626947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.626896    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.628272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.628254    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.628791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.628771    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.680631 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.680590    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.680774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.680733    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jdrs\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\") pod \"e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") " pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:32.772656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:32.772554    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:33.199329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.199292    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:34:33.236429 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:33.236395    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e162ea_0824_4415_b003_b10224a62149.slice/crio-9f7def421ed1dfe5c113fdcdf7c7529fe4fa48f9fb184ed10466485476aa2785 WatchSource:0}: Error finding container 9f7def421ed1dfe5c113fdcdf7c7529fe4fa48f9fb184ed10466485476aa2785: Status 404 returned error can't find the container with id 9f7def421ed1dfe5c113fdcdf7c7529fe4fa48f9fb184ed10466485476aa2785
Nov 02 21:34:33.239339 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.239299    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:33.239431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.239373    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:33.239431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.239402    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:33.331394 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.331361    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 21:34:33.335577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.335543    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.393000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.390821    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 21:34:33.434837 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.434793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" event={"ID":"03e162ea-0824-4415-b003-b10224a62149","Type":"ContainerStarted","Data":"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"}
Nov 02 21:34:33.435029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.434940    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" event={"ID":"03e162ea-0824-4415-b003-b10224a62149","Type":"ContainerStarted","Data":"9f7def421ed1dfe5c113fdcdf7c7529fe4fa48f9fb184ed10466485476aa2785"}
Nov 02 21:34:33.439716 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.439681    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.439905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.439722    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q9xx\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.439905 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.439751    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.439942    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.440058    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.440185    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.440232    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.440268    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.440318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.440292    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542030    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542070    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q9xx\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542091    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542269    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542349    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542437    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542487    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542532    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.542710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542575    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.543101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542726    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.543101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.543101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.542818    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.543265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.543104    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.543316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.543270    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.544946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.544919    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.545310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.545289    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.556161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.556113    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podStartSLOduration=1.556100872 podStartE2EDuration="1.556100872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:33.554144434 +0000 UTC m=+1303.097936349" watchObservedRunningTime="2026-11-02 21:34:33.556100872 +0000 UTC m=+1303.099892787"
Nov 02 21:34:33.596079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.596038    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.596243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.596124    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q9xx\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\") pod \"e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") " pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.656266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.656231    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:33.772953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:33.772916    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:34.079933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.079902    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 21:34:34.080146 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:34:34.080123    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7c87f6_5c46_4a9c_8d8d_c38a65e94b91.slice/crio-32bb2b5e194c0ed0563612ebfb0ac849ab27bd9f38cca88910df7947703f74c4 WatchSource:0}: Error finding container 32bb2b5e194c0ed0563612ebfb0ac849ab27bd9f38cca88910df7947703f74c4: Status 404 returned error can't find the container with id 32bb2b5e194c0ed0563612ebfb0ac849ab27bd9f38cca88910df7947703f74c4
Nov 02 21:34:34.083283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.083253    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:34.083366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.083318    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:34.083366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.083362    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:34:34.449948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.449906    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" event={"ID":"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91","Type":"ContainerStarted","Data":"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"}
Nov 02 21:34:34.450126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.449967    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" event={"ID":"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91","Type":"ContainerStarted","Data":"32bb2b5e194c0ed0563612ebfb0ac849ab27bd9f38cca88910df7947703f74c4"}
Nov 02 21:34:34.566152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.566101    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podStartSLOduration=1.566085953 podStartE2EDuration="1.566085953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:34.563173848 +0000 UTC m=+1304.106965764" watchObservedRunningTime="2026-11-02 21:34:34.566085953 +0000 UTC m=+1304.109877870"
Nov 02 21:34:34.656364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.656331    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:34.774895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.774799    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:34.774895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:34.774883    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:35.719695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:35.719654    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:35.882910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:35.876063    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:35.882910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:35.804804    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:35.882910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:35.876211    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:36.648030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.647986    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:36.659910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.658151    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:36.659910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.658211    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:36.776376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.776333    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:36.777005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.776962    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:36.804873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:36.803971    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:34:37.334879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.330973    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:37.395175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.393945    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:37.655071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.655028    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:37.656491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.656464    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:34:37.658549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.657623    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:37.658781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.658756    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:37.810881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.804947    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:37.810881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.805405    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:34:37.810881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.807311    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:37.810881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:37.807358    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:38.315871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.290579    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:34:38.347880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.335030    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:38.347880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.335456    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:34:38.501168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.501080    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:34:38.545230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.504028    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:38.545230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.529948    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:38.545230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.532626    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:34:38.705074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.676625    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:34:38.728778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.728740    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:38.729177 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.729049    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:38.813471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.813420    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:38.880873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.872047    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:38.880873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.872148    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:38.935304 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:38.935256    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.066649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.066543    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.337967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.320105    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.337967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.322293    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:34:39.341417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.341378    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.546877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.546133    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:34:39.547285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.547247    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:34:39.630729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.630634    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:34:39.670783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.670747    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:39.671066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.671042    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:39.690384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.690329    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:34:39.832028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.831991    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:39.852963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.852918    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:39.853253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.835913    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.881815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.881728    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:34:39.900176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.900011    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:34:39.900176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.900087    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.951876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.948176    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:39.951876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:39.948584    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:34:40.123869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.110621    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:34:40.123869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.111162    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:34:40.123869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.113626    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:34:40.201289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.201256    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:34:40.231034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.223066    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:34:40.231034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.225047    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:34:40.233995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.233970    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:34:40.285138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.278108    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": context deadline exceeded"
Nov 02 21:34:40.380067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.376612    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:34:40.380067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.378998    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:34:40.676054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.676017    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:40.676249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.676083    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:40.782951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.782801    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:40.783442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:40.783406    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:41.677902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:41.675023    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:41.677902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:41.675089    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:41.773007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:41.772961    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:34:41.773179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:41.773028    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 21:34:42.657122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:42.657080    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:34:42.657345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:42.657146    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 02 21:34:43.815095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:43.815033    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:34:43.815601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:43.815112    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:34:43.844257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:43.844221    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:43.845267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:43.845240    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:43.846956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:43.846921    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 21:34:44.378601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:44.378568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:44.378925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:44.378903    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:44.379915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:44.379894    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:34:52.260772 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:34:52.260730    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:34:56.346167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:56.346128    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:34:56.346621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:34:56.346327    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy" containerID="cri-o://4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7" gracePeriod=30
Nov 02 21:35:01.610192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.610123    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:35:01.716977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.716935    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717093    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717199    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717250    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717298    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717381    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717436    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717463    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:01.717502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717477    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:01.717502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717494    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717538    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\") pod \"8492a874-e5b6-46a9-9d09-cc75809f58e8\" (UID: \"8492a874-e5b6-46a9-9d09-cc75809f58e8\") "
Nov 02 21:35:01.717658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717608    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:01.717822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717779    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:01.717822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.717788    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:01.718247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.718223    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.718349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.718253    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.718349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.718271    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.718349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.718285    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8492a874-e5b6-46a9-9d09-cc75809f58e8-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.718349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.718300    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.719787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.719736    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:35:01.719787 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.719736    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:01.719987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.719871    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "kube-api-access-bgv4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:01.720119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.720102    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy" pod "8492a874-e5b6-46a9-9d09-cc75809f58e8" (UID: "8492a874-e5b6-46a9-9d09-cc75809f58e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:01.819496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.819461    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.819496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.819497    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.819720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.819511    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8492a874-e5b6-46a9-9d09-cc75809f58e8-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:01.819720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:01.819523    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgv4l\" (UniqueName: \"kubernetes.io/projected/8492a874-e5b6-46a9-9d09-cc75809f58e8-kube-api-access-bgv4l\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:02.016682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.016649    4888 generic.go:356] "Generic (PLEG): container finished" podID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerID="4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7" exitCode=0
Nov 02 21:35:02.016895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.016704    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"
Nov 02 21:35:02.016895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.016739    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" event={"ID":"8492a874-e5b6-46a9-9d09-cc75809f58e8","Type":"ContainerDied","Data":"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"}
Nov 02 21:35:02.016895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.016832    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7" event={"ID":"8492a874-e5b6-46a9-9d09-cc75809f58e8","Type":"ContainerDied","Data":"f9a2df12afbc79068f9a11e0425be71b05dee1a0e3ac4a9f2cd4d42e8aa0fcee"}
Nov 02 21:35:02.016895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.016885    4888 scope.go:122] "RemoveContainer" containerID="4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"
Nov 02 21:35:02.030806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.030759    4888 scope.go:122] "RemoveContainer" containerID="4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"
Nov 02 21:35:02.031221 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:02.031195    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7\": container with ID starting with 4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7 not found: ID does not exist" containerID="4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"
Nov 02 21:35:02.031318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.031237    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7"} err="failed to get container status \"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7\": rpc error: code = NotFound desc = could not find container \"4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7\": container with ID starting with 4ec3998f397cc9e4919a110bb8d4cf20e1f03972fa6ed621ed8e3dd94b05eef7 not found: ID does not exist"
Nov 02 21:35:02.173867 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.173825    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:35:02.177249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.177218    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7"]
Nov 02 21:35:02.600719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:02.600684    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" path="/var/lib/kubelet/pods/8492a874-e5b6-46a9-9d09-cc75809f58e8/volumes"
Nov 02 21:35:03.406278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.406243    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7155-29bcda"]
Nov 02 21:35:03.406788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.406773    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy"
Nov 02 21:35:03.406867 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.406790    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy"
Nov 02 21:35:03.406951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.406941    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8492a874-e5b6-46a9-9d09-cc75809f58e8" containerName="istio-proxy"
Nov 02 21:35:03.411491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.411464    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:03.457463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.457414    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th9zj\" (UniqueName: \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\") pod \"test-curl-7155-29bcda\" (UID: \"a3fee7d9-a281-46a8-91f3-29003a053591\") " pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:03.461518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.461482    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7155-29bcda"]
Nov 02 21:35:03.504691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.504656    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9vlh9\""
Nov 02 21:35:03.559187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.559151    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th9zj\" (UniqueName: \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\") pod \"test-curl-7155-29bcda\" (UID: \"a3fee7d9-a281-46a8-91f3-29003a053591\") " pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:03.615543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.615504    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th9zj\" (UniqueName: \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\") pod \"test-curl-7155-29bcda\" (UID: \"a3fee7d9-a281-46a8-91f3-29003a053591\") " pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:03.723480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:03.723383    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:04.090195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:04.090166    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7155-29bcda"]
Nov 02 21:35:04.091183 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:35:04.091143    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3fee7d9_a281_46a8_91f3_29003a053591.slice/crio-42fc748adcbd1ed2077c019f1cf533e8509cf1c75a198985fdb144c6032e9d86 WatchSource:0}: Error finding container 42fc748adcbd1ed2077c019f1cf533e8509cf1c75a198985fdb144c6032e9d86: Status 404 returned error can't find the container with id 42fc748adcbd1ed2077c019f1cf533e8509cf1c75a198985fdb144c6032e9d86
Nov 02 21:35:04.093278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:04.093262    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:35:05.040062 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:05.040021    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7155-29bcda" event={"ID":"a3fee7d9-a281-46a8-91f3-29003a053591","Type":"ContainerStarted","Data":"42fc748adcbd1ed2077c019f1cf533e8509cf1c75a198985fdb144c6032e9d86"}
Nov 02 21:35:08.085995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:08.085960    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7155-29bcda" event={"ID":"a3fee7d9-a281-46a8-91f3-29003a053591","Type":"ContainerStarted","Data":"e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b"}
Nov 02 21:35:08.193103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:08.193046    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7155-29bcda" podStartSLOduration=2.324272261 podStartE2EDuration="5.193027375s" totalImagesPullingTime="2.868755114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:08.19255613 +0000 UTC m=+1337.736348046" watchObservedRunningTime="2026-11-02 21:35:08.193027375 +0000 UTC m=+1337.736819291"
Nov 02 21:35:09.590387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.590345    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7155-29bcda"]
Nov 02 21:35:09.595242 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.595208    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7155-29bcda"]
Nov 02 21:35:09.749474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.749436    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7155-f2ba4b"]
Nov 02 21:35:09.750086 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.750069    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" containerName="test-curl-7155-29bcda"
Nov 02 21:35:09.750138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.750088    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" containerName="test-curl-7155-29bcda"
Nov 02 21:35:09.750243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.750218    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" containerName="test-curl-7155-29bcda"
Nov 02 21:35:09.753772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.753750    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:09.812317 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.812284    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7155-f2ba4b"]
Nov 02 21:35:09.857354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.857262    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfzgh\" (UniqueName: \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\") pod \"test-curl-7155-f2ba4b\" (UID: \"de92f5d7-de58-4860-8bfb-bad9b2101fe3\") " pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:09.958475 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:09.958435    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfzgh\" (UniqueName: \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\") pod \"test-curl-7155-f2ba4b\" (UID: \"de92f5d7-de58-4860-8bfb-bad9b2101fe3\") " pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:10.013989 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:10.013960    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfzgh\" (UniqueName: \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\") pod \"test-curl-7155-f2ba4b\" (UID: \"de92f5d7-de58-4860-8bfb-bad9b2101fe3\") " pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:10.066611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:10.066569    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:10.099408 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:10.099342    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7155-29bcda" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" containerName="test-curl-7155-29bcda" containerID="cri-o://e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b" gracePeriod=2
Nov 02 21:35:10.149888 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:10.148188    4888 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7155-29bcda\" is forbidden: User \"system:node:ip-10-0-130-83.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-83.us-west-2.compute.internal' and this object" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:10.436860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:10.436823    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7155-f2ba4b"]
Nov 02 21:35:10.437541 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:35:10.437516    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde92f5d7_de58_4860_8bfb_bad9b2101fe3.slice/crio-69bcd0623b65fc57b850bf82418852ef4433c8173a2eb796805609c9a7276782 WatchSource:0}: Error finding container 69bcd0623b65fc57b850bf82418852ef4433c8173a2eb796805609c9a7276782: Status 404 returned error can't find the container with id 69bcd0623b65fc57b850bf82418852ef4433c8173a2eb796805609c9a7276782
Nov 02 21:35:10.645574 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:10.645532    4888 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7155-29bcda\" is forbidden: User \"system:node:ip-10-0-130-83.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-83.us-west-2.compute.internal' and this object" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:11.113543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:11.113481    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7155-f2ba4b" event={"ID":"de92f5d7-de58-4860-8bfb-bad9b2101fe3","Type":"ContainerStarted","Data":"c637456e9e1bed4b1662aa9dc4addb5dcf857406a8f424352fe5df5953b78a21"}
Nov 02 21:35:11.113543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:11.113549    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7155-f2ba4b" event={"ID":"de92f5d7-de58-4860-8bfb-bad9b2101fe3","Type":"ContainerStarted","Data":"69bcd0623b65fc57b850bf82418852ef4433c8173a2eb796805609c9a7276782"}
Nov 02 21:35:11.222217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:11.222163    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7155-f2ba4b" podStartSLOduration=2.222149183 podStartE2EDuration="2.222149183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:11.219116554 +0000 UTC m=+1340.762908471" watchObservedRunningTime="2026-11-02 21:35:11.222149183 +0000 UTC m=+1340.765941098"
Nov 02 21:35:12.504869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.502972    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:12.525873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.522431    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\" (UniqueName: \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\") pod \"a3fee7d9-a281-46a8-91f3-29003a053591\" (UID: \"a3fee7d9-a281-46a8-91f3-29003a053591\") "
Nov 02 21:35:12.532018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.531963    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj" pod "a3fee7d9-a281-46a8-91f3-29003a053591" (UID: "a3fee7d9-a281-46a8-91f3-29003a053591"). InnerVolumeSpecName "kube-api-access-th9zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:12.602829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.602790    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3fee7d9-a281-46a8-91f3-29003a053591" path="/var/lib/kubelet/pods/a3fee7d9-a281-46a8-91f3-29003a053591/volumes"
Nov 02 21:35:12.623990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.623953    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th9zj\" (UniqueName: \"kubernetes.io/projected/a3fee7d9-a281-46a8-91f3-29003a053591-kube-api-access-th9zj\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:12.890874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.890024    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7155-f2ba4b"]
Nov 02 21:35:12.947880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:12.941975    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7155-f2ba4b"]
Nov 02 21:35:13.346888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.345218    4888 generic.go:356] "Generic (PLEG): container finished" podID="a3fee7d9-a281-46a8-91f3-29003a053591" containerID="e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b" exitCode=137
Nov 02 21:35:13.346888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.345528    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7155-f2ba4b" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" containerName="test-curl-7155-f2ba4b" containerID="cri-o://c637456e9e1bed4b1662aa9dc4addb5dcf857406a8f424352fe5df5953b78a21" gracePeriod=2
Nov 02 21:35:13.346888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.345919    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-29bcda"
Nov 02 21:35:13.346888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.346450    4888 scope.go:122] "RemoveContainer" containerID="e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b"
Nov 02 21:35:13.409427 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:13.401261    4888 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7155-f2ba4b\" is forbidden: User \"system:node:ip-10-0-130-83.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-83.us-west-2.compute.internal' and this object" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:13.447232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.439157    4888 scope.go:122] "RemoveContainer" containerID="e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b"
Nov 02 21:35:13.447783 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:13.447599    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b\": container with ID starting with e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b not found: ID does not exist" containerID="e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b"
Nov 02 21:35:13.447783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.447646    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b"} err="failed to get container status \"e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b\": rpc error: code = NotFound desc = could not find container \"e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b\": container with ID starting with e64b287b7a75fc43114e6ca48e9833c76ec5656048e7ff4f9800467800bbfb1b not found: ID does not exist"
Nov 02 21:35:13.462145 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:13.462092    4888 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7155-f2ba4b\" is forbidden: User \"system:node:ip-10-0-130-83.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-83.us-west-2.compute.internal' and this object" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:13.936874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.933425    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:35:13.936874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.934283    4888 cpu_manager.go:414] "containerMap: removing container" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" containerName="test-curl-7155-f2ba4b"
Nov 02 21:35:13.936874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.934306    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" containerName="test-curl-7155-f2ba4b"
Nov 02 21:35:13.936874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.934500    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" containerName="test-curl-7155-f2ba4b"
Nov 02 21:35:13.968654 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:13.968621    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.148895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.136251    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c39c4123-openshift-default-dockercfg-t7dqf\""
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.171642    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.171713    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nppnn\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.171751    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.171821    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.171971    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.172046    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.172073    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.172102    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.172419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.172153    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.254932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.247911    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:14.282279    4888 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7155-f2ba4b\" is forbidden: User \"system:node:ip-10-0-130-83.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-83.us-west-2.compute.internal' and this object" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.283985    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nppnn\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284034    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284097    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284233    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284258    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284283    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284324    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.284536    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.285293    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.289949    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.301962    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.303709    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.303998    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.357876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.334969    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.364548 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.285835    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.436873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.398739    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:35:14.519623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.516928    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:14.580035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.579970    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nppnn\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.588600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.588523    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.592879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.590505    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.685229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.684110    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:14.741034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.740997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\") pod \"e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") " pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:14.987911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:14.980114    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:15.236114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.236067    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:35:15.262955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.262796    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.336539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.316022    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.390894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.339598    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:35:15.417912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.397523    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.423411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.423368    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.426665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.426636    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:35:15.433878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.432502    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.457074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.437466    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:15.686953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.686907    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.689770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.689743    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:15.723798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.722589    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:15.745756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.735761    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:15.745756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:15.735826    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:35:16.170399 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.170348    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:16.210170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.196406    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:16.271060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.270950    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:16.272219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.272196    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:35:16.428949 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.425070    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:16.476872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.475955    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:35:16.476872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.476024    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:16.476872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.448661    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:35:16.476872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.434138    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:35:16.541930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.429222    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:16.565605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.542565    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:35:16.748780 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:16.731942    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:17.233872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.191050    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.233872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.196551    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.233872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.223090    4888 generic.go:356] "Generic (PLEG): container finished" podID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" containerID="c637456e9e1bed4b1662aa9dc4addb5dcf857406a8f424352fe5df5953b78a21" exitCode=137
Nov 02 21:35:17.233872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.226842    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.540056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.539948    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.656874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.558148    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.663959 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.558382    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:17.724876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.676612    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/health\": context deadline exceeded"
Nov 02 21:35:17.724876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.612720    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:17.724876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.677101    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:17.724876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.677461    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:17.771873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.584371    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:35:17.930937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:17.776722    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:17.990274 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:17.990225    4888 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecc7e64_b1b1_4356_bbbf_007f6348e168.slice/crio-f6de2392059d15a587424a3e4ab2e352720cd17fe11c7a456d119f9f31fbe272\": RecentStats: unable to find data in memory cache]"
Nov 02 21:35:18.167072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.167024    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.227997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.206363    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.238161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.212207    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:18.297033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.296935    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.396079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.396041    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:35:18.595515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.526223    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerStarted","Data":"f6de2392059d15a587424a3e4ab2e352720cd17fe11c7a456d119f9f31fbe272"}
Nov 02 21:35:18.595515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.529143    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69bcd0623b65fc57b850bf82418852ef4433c8173a2eb796805609c9a7276782"
Nov 02 21:35:18.621872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.597382    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:18.745155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.619835    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.745546 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.698539    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.745862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.698590    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.746107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.698645    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:18.962054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:18.957588    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:19.040061 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.040027    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:19.112480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.112397    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\" (UniqueName: \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\") pod \"de92f5d7-de58-4860-8bfb-bad9b2101fe3\" (UID: \"de92f5d7-de58-4860-8bfb-bad9b2101fe3\") "
Nov 02 21:35:19.135666 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.135629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh" pod "de92f5d7-de58-4860-8bfb-bad9b2101fe3" (UID: "de92f5d7-de58-4860-8bfb-bad9b2101fe3"). InnerVolumeSpecName "kube-api-access-tfzgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:19.162222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.162190    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:35:19.176809 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.176778    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:35:19.177055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.174642    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:35:19.178434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.178248    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:35:19.208184 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.207923    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:19.211885 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.211859    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:35:19.256075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.256039    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfzgh\" (UniqueName: \"kubernetes.io/projected/de92f5d7-de58-4860-8bfb-bad9b2101fe3-kube-api-access-tfzgh\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:35:19.642630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.639695    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7155-f2ba4b"
Nov 02 21:35:19.642630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.640042    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerStarted","Data":"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"}
Nov 02 21:35:19.745876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.745818    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:35:19.759919 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.748023    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:19.853522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:19.853447    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podStartSLOduration=6.853427351 podStartE2EDuration="6.853427351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:19.781986726 +0000 UTC m=+1349.325778643" watchObservedRunningTime="2026-11-02 21:35:19.853427351 +0000 UTC m=+1349.397219269"
Nov 02 21:35:20.002697 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.001201    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:20.125874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.125012    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:35:20.125874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.125073    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:35:20.125874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.125715    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": context deadline exceeded"
Nov 02 21:35:20.317926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.305748    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:35:20.381949 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.381065    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:20.525900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.525532    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:20.637929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.631674    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de92f5d7-de58-4860-8bfb-bad9b2101fe3" path="/var/lib/kubelet/pods/de92f5d7-de58-4860-8bfb-bad9b2101fe3/volumes"
Nov 02 21:35:20.739141 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.739087    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:20.784872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:20.784281    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:35:21.218389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:21.218345    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:21.700546 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:21.700510    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:21.701611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:21.701594    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:35:22.261300 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:22.261264    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:35:27.887900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:27.887860    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:35:27.894342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:27.894316    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.021872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:27.979268    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.021872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:27.979328    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4qr6\" (UniqueName: \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.033966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.033935    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2ccb9fb8\""
Nov 02 21:35:28.071874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.042640    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2ccb9fb8-dockercfg-stdwm\""
Nov 02 21:35:28.106880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.081885    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.149334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.087997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.149626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.149234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4qr6\" (UniqueName: \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.151594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.151559    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:35:28.204098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.204068    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4qr6\" (UniqueName: \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\") pod \"payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") " pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.210151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.210108    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.589975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.589936    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:35:28.804249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.804213    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" event={"ID":"0a8b18e7-ab31-4f2b-a170-45040d0ef281","Type":"ContainerStarted","Data":"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"}
Nov 02 21:35:28.804434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.804264    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" event={"ID":"0a8b18e7-ab31-4f2b-a170-45040d0ef281","Type":"ContainerStarted","Data":"f47678f4964dffdd84003b10068d2faee154cbe1f6f1dd724bcb88335a17b0f8"}
Nov 02 21:35:28.804434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.804380    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:28.914104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:28.914047    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" podStartSLOduration=1.914032864 podStartE2EDuration="1.914032864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:28.910575793 +0000 UTC m=+1358.454367710" watchObservedRunningTime="2026-11-02 21:35:28.914032864 +0000 UTC m=+1358.457824780"
Nov 02 21:35:34.057862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.057810    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:35:34.062274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.062239    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.125509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.125457    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.125784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.125764    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.125981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.125961    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126186    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126525    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126675    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126811    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.126969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.126953    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnvk\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.150975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.150925    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:35:34.156574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.156542    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.236608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.236564    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.236922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.236902    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.237082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237066    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.237209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237195    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.237340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237323    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnvk\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.237463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237449    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.237650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237635    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.237806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.237781    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.239020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.238995    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4qdm\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.239251 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.239235    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.239383 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.239368    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.239521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.239506    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.239705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.239691    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.239830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.239814    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.244986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.244947    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.264839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.263516    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.264839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.264312    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.264839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.264549    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.265163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.265110    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.267823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.265348    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.268029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.267914    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.268029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.267995    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.268339 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.268295    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.273312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.273283    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.347299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.346750    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.350300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.349718    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnvk\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.350300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.350157    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.350300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.350204    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:35:34.372057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.371973    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.372374 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.372353    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.372579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.372551    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.372891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.372837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4qdm\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.373185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.373155    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.373333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.373316    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.373445 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.373431    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.373678 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.373652    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.373978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.373874    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.430635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.382292    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:34.430635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.423952    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:35:34.441691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.441641    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.461900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.458029    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.488745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.488662    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.489716 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.489643    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.490356 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.490332    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.493907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.491256    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.493907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.492603    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.493907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.493025    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.532909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.532836    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4qdm\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\") pod \"e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") " pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.575120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.575082    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:34.955271 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:34.955233    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:35:35.028034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.027991    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:35:35.040413 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:35:35.040375    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c627dd_d460_4e2f_8e73_8dbf3d391aec.slice/crio-aa82125ffa4b9e4f3d86ca7acba04f5b7084d75c914701e806cc56140aef3c19 WatchSource:0}: Error finding container aa82125ffa4b9e4f3d86ca7acba04f5b7084d75c914701e806cc56140aef3c19: Status 404 returned error can't find the container with id aa82125ffa4b9e4f3d86ca7acba04f5b7084d75c914701e806cc56140aef3c19
Nov 02 21:35:35.051193 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:35:35.050994    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fddb0e6_dcad_40fd_a480_a8be9ceb3874.slice/crio-9e53b1e976e2df1590c02405b60e330dbb93e9697bbb125acfcbcecda8ee8f44 WatchSource:0}: Error finding container 9e53b1e976e2df1590c02405b60e330dbb93e9697bbb125acfcbcecda8ee8f44: Status 404 returned error can't find the container with id 9e53b1e976e2df1590c02405b60e330dbb93e9697bbb125acfcbcecda8ee8f44
Nov 02 21:35:35.066504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.066451    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.067215 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.067186    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.067364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.067346    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.073908 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.073784    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.073908 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.073864    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.073908 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.073895    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:35.877514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.877472    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" event={"ID":"64c627dd-d460-4e2f-8e73-8dbf3d391aec","Type":"ContainerStarted","Data":"e0fd34a94fe62e4245f7e87fb23f16aceeae76a57ca56f1f7a4205436f3b4dd0"}
Nov 02 21:35:35.877692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.877549    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" event={"ID":"64c627dd-d460-4e2f-8e73-8dbf3d391aec","Type":"ContainerStarted","Data":"aa82125ffa4b9e4f3d86ca7acba04f5b7084d75c914701e806cc56140aef3c19"}
Nov 02 21:35:35.879649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.879622    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" event={"ID":"1fddb0e6-dcad-40fd-a480-a8be9ceb3874","Type":"ContainerStarted","Data":"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"}
Nov 02 21:35:35.879756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:35.879667    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" event={"ID":"1fddb0e6-dcad-40fd-a480-a8be9ceb3874","Type":"ContainerStarted","Data":"9e53b1e976e2df1590c02405b60e330dbb93e9697bbb125acfcbcecda8ee8f44"}
Nov 02 21:35:36.094309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.094238    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podStartSLOduration=2.094217542 podStartE2EDuration="2.094217542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:36.044607935 +0000 UTC m=+1365.588399852" watchObservedRunningTime="2026-11-02 21:35:36.094217542 +0000 UTC m=+1365.638009460"
Nov 02 21:35:36.169873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.167772    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podStartSLOduration=2.167748237 podStartE2EDuration="2.167748237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:36.160656783 +0000 UTC m=+1365.704448699" watchObservedRunningTime="2026-11-02 21:35:36.167748237 +0000 UTC m=+1365.711540154"
Nov 02 21:35:36.448287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.446005    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:36.463620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.463293    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:36.463620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.463359    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:36.605747 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.605717    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:36.606344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.606316    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:36.606536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:36.606512    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:37.138030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.137980    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:37.378912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.300762    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:35:37.411478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.387205    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:37.411478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.387270    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:37.411478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.387627    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:35:37.411478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.387722    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.450330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.439679    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4dff0429\""
Nov 02 21:35:37.456608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.454954    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4dff0429-dockercfg-8cgtn\""
Nov 02 21:35:37.486900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.471973    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:37.486900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.473243    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsgsg\" (UniqueName: \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.486900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.473293    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.513338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.495078    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:35:37.547898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.546317    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:35:37.578936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.578742    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:37.578936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.578807    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:37.610398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.598017    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsgsg\" (UniqueName: \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.610398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.598066    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.638759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.638724    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.687947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.684182    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:37.744885 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.734023    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsgsg\" (UniqueName: \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\") pod \"payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:37.760600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.756957    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:37.760600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.758319    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:35:37.760600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.759564    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:35:37.802142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:37.802103    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:38.167879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.166967    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:38.211802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.211751    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:38.385873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.384383    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:38.385873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.384443    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:38.484543 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.482658    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:38.544557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.542980    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:38.544557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.543409    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:38.879934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.873093    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:38.879934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.873173    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:38.886709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.886659    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:38.950009 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.949975    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:38.951560 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.951541    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:38.999278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:38.999251    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:39.102204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.097226    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:35:39.208626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.207894    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" event={"ID":"83c02484-883f-41d8-8a6c-84c4994bb95a","Type":"ContainerStarted","Data":"4eb38b75fe7d1c0dea71dba0040a967c7f4c7d9450bee852b898cc60895984bb"}
Nov 02 21:35:39.383879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.383832    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:39.384132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.384102    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:39.587879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.587424    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:39.587879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.587484    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:39.865361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:39.865252    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:35:40.276240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.276204    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" event={"ID":"83c02484-883f-41d8-8a6c-84c4994bb95a","Type":"ContainerStarted","Data":"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"}
Nov 02 21:35:40.276634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.276261    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:40.383115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.383076    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:40.383317 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.383144    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:40.402306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.402252    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" podStartSLOduration=3.402237013 podStartE2EDuration="3.402237013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:40.399236524 +0000 UTC m=+1369.943028436" watchObservedRunningTime="2026-11-02 21:35:40.402237013 +0000 UTC m=+1369.946028926"
Nov 02 21:35:40.668872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.666009    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:40.668872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:40.666072    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:41.383989 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:41.383952    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:41.384528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:41.384013    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:41.576139 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:41.576104    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:41.576318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:41.576155    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:42.383469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:42.383435    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:42.383648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:42.383500    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:42.575969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:42.575928    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:42.576490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:42.575997    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:43.383498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:43.383402    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:35:43.383498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:43.383466    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 21:35:43.575823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:43.575778    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:35:43.576036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:43.575888    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 21:35:44.388629 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.388595    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:44.389020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.388989    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:44.389892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.389833    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:35:44.580494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.580464    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:44.580980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.580772    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:44.581724 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:44.581705    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:35:46.856638 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:46.852062    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:35:46.861271 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:46.861242    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:46.882387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:46.882352    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:35:46.942915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:46.942872    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s69j\" (UniqueName: \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:46.986883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:46.983880    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.026413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.026365    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c7cd2d37\""
Nov 02 21:35:47.027141 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.027115    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c7cd2d37-dockercfg-62gwz\""
Nov 02 21:35:47.151982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.150021    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s69j\" (UniqueName: \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.151982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.150080    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.151982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.151494    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.247677 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.247640    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s69j\" (UniqueName: \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\") pod \"payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.338301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.338249    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:47.709955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.709919    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:35:47.711151 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:35:47.711122    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d61f536_8217_413b_a18b_9db80ca82c32.slice/crio-c84abafa7f831895f79a38188f2992bf328077e39399e5f9385be64c12e72495 WatchSource:0}: Error finding container c84abafa7f831895f79a38188f2992bf328077e39399e5f9385be64c12e72495: Status 404 returned error can't find the container with id c84abafa7f831895f79a38188f2992bf328077e39399e5f9385be64c12e72495
Nov 02 21:35:47.715608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.715580    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:35:47.722535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.722511    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.757278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.757232    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765515    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765559    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765600    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765633    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765725    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765752    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765777    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrzm8\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765814    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765866    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.767351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.765928    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.808810 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.808774    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:35:47.815548 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.815507    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:35:47.867362 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867330    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867376    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrzm8\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867543    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867593    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867637    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5s87\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867668    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867702    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867750    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.867820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867783    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.867940    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868119    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868145    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868179    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868197    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868278    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868357    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.868633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868408    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868440    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868460    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.868633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868480    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.868633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.868523    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.869929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.869569    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.869929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.869838    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.871143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.871111    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.871503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.871482    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.922334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.922299    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrzm8\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.922541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.922428    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:47.970266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970187    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970320    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970423    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5s87\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970451    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970478    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970529    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970599    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970633    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970636    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970693    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970791    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.970963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.970926    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.971311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.971283    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.972917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.972893    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:47.973031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:47.973014    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:48.025258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.025224    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:48.025423 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.025339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5s87\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\") pod \"e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") " pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:48.067522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.067480    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:48.094494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.094463    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:48.431648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.431605    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" event={"ID":"2d61f536-8217-413b-a18b-9db80ca82c32","Type":"ContainerStarted","Data":"8735cc8d63427c364fc8191dc19e0c0862e14df509a8fce76fd145dfe2349171"}
Nov 02 21:35:48.431812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.431662    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" event={"ID":"2d61f536-8217-413b-a18b-9db80ca82c32","Type":"ContainerStarted","Data":"c84abafa7f831895f79a38188f2992bf328077e39399e5f9385be64c12e72495"}
Nov 02 21:35:48.431917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.431861    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:35:48.540120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.539878    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:35:48.555878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.545887    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.565910    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.566015    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.566055    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.567668    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.567749    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:48.567869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:48.567792    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:35:49.440279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.440234    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" event={"ID":"547be484-ab43-4f26-9f3a-f44c5674727b","Type":"ContainerStarted","Data":"4daee9dd5131de3584b5b700710124699e4c52fe3617f88eb2c64da166f74113"}
Nov 02 21:35:49.440726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.440341    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" event={"ID":"547be484-ab43-4f26-9f3a-f44c5674727b","Type":"ContainerStarted","Data":"65812e59faba5c57a5ea4a7dbc6201d0cc13ce40aec55ac97de3f94ac4bf123c"}
Nov 02 21:35:49.442387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.442349    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" event={"ID":"d072b7ab-499d-4b3a-8826-8d35d52ffaf0","Type":"ContainerStarted","Data":"10356d08420322b3795eae4dda2053b5d9402fc6c68a6e70812f93092169ba19"}
Nov 02 21:35:49.442540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.442424    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" event={"ID":"d072b7ab-499d-4b3a-8826-8d35d52ffaf0","Type":"ContainerStarted","Data":"04bcd798c588487e4ad8cd3d520bc46951dbb55189c5c519e7c434e1b42706d8"}
Nov 02 21:35:49.553887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.553794    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" podStartSLOduration=3.553781855 podStartE2EDuration="3.553781855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:48.613416008 +0000 UTC m=+1378.157207925" watchObservedRunningTime="2026-11-02 21:35:49.553781855 +0000 UTC m=+1379.097573771"
Nov 02 21:35:49.554121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.554095    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podStartSLOduration=2.554090905 podStartE2EDuration="2.554090905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:49.55310152 +0000 UTC m=+1379.096893435" watchObservedRunningTime="2026-11-02 21:35:49.554090905 +0000 UTC m=+1379.097882821"
Nov 02 21:35:49.664464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:49.664383    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podStartSLOduration=2.664363566 podStartE2EDuration="2.664363566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:49.662452408 +0000 UTC m=+1379.206244334" watchObservedRunningTime="2026-11-02 21:35:49.664363566 +0000 UTC m=+1379.208155485"
Nov 02 21:35:50.068364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.068328    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:35:50.069891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.069865    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:50.070007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.069911    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:50.095288 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.095258    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:35:50.097170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.097136    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:50.097273 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:50.097184    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:51.068658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:51.068572    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:51.068658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:51.068626    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:51.095281 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:51.095247    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:51.095458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:51.095301    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:51.388015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:51.387930    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:35:52.085090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.085054    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:52.085735 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.085682    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:52.108913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.108877    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:52.109184 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.109154    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:52.522264 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:35:52.522218    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:35:52.545030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.544960    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:52.569297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.569253    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:52.631973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.631930    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:52.738974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.738819    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:52.742362 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.739244    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:52.796058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.795956    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:52.817574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:52.817523    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:53.012365 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.012319    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:53.069509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.069425    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:53.069760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.069734    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:53.097708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.097665    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:53.098412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.098377    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:53.196382 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.196271    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:35:53.215997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.215949    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:35:53.510877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.496715    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:35:53.570900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.568538    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:35:53.570900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.569721    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:35:53.570900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.570661    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:53.571219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.570948    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:35:53.749996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.749947    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:53.750692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.750658    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:53.751194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.751174    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:53.751489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.751471    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:35:53.803699 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.803594    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:53.850094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:53.820150    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:35:54.044006 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.043964    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:54.073552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.073459    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:54.073803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.073772    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:54.074065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.074048    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:54.219317 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.219172    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:54.220370 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.220348    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:35:54.353199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.325413    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:54.353199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.325493    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:54.608935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.602190    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.608935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.602595    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.608935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.604220    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.608935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.604540    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:35:54.817875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.781995    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.817875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.782309    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.817875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.816934    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:35:54.916468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.916432    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:35:54.916925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:54.916745    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.066910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.054344    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.142828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.111102    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:55.142828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.111163    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:55.142828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.111248    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:55.142828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.111275    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:55.224084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.223972    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.246895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.224620    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:35:55.615093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.614983    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.616798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.616763    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.618134 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.618096    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:35:55.797873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.796045    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:55.797873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.797568    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:55.833951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:55.833901    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.070272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.070233    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:56.070527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.070495    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:56.070768 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.070692    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.107147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.107111    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:56.107450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.107404    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:56.242001 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.241959    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.243080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.243058    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:35:56.252318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.252248    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.627879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.616939    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:35:56.627879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.617772    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:35:56.643873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.633386    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.802390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.802335    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.832901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.809558    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:56.834872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:56.834800    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.076471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.076422    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.141307 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.087005    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:57.141579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.141540    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:57.141701 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.109180    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:57.141821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.141795    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:57.167981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.161950    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.267868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.267037    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:35:57.364943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.290942    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.366016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.365977    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:35:57.366224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.366194    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.620075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.619968    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.626948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.626906    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:35:57.646990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.634159    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Nov 02 21:35:57.673987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.673934    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/health\": context deadline exceeded"
Nov 02 21:35:57.714983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.711197    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:35:57.852874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.839105    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:35:57.852874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.839665    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:35:57.852874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.839839    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:35:57.890213 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:57.890081    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 21:35:58.091916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.091872    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:35:58.093452 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.093417    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:35:58.094910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.094868    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:58.095092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.095067    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:58.095983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.095163    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:58.095983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.095202    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:58.099873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.099829    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:35:58.201391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.191086    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:35:58.503449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.503357    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:35:58.507418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.507389    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 21:35:58.509842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:58.509820    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:35:59.080664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:59.080152    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:35:59.080664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:59.080224    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 21:35:59.095239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:59.095198    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 21:35:59.095545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:59.095500    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 21:35:59.445900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:35:59.445669    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:36:00.478518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.478483    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:36:00.479948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.479877    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:36:00.481367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.481341    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:36:00.531364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.531335    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:36:00.532345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.532314    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:36:00.533400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:00.533373    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:36:07.067709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.067670    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:36:07.073679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.073657    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.119256 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.119217    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.119579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.119538    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.119840 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.119821    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.120012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.119898    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt267\" (UniqueName: \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.123780 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.123757    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:36:07.167084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.167055    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c39c4123-serving-cert\""
Nov 02 21:36:07.221640 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.221597    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.221828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.221693    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt267\" (UniqueName: \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.221909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.221841    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.222021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.222003    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.222146 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:07.222123    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-serving-cert: secret "maas-api-e2e-ait-c39c4123-serving-cert" not found
Nov 02 21:36:07.222210 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:07.222196    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls podName:191f2736-d3e9-4696-962e-fec0e31de279 nodeName:}" failed. No retries permitted until 2026-11-02 21:36:07.722181883 +0000 UTC m=+1397.265973795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls") pod "maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" (UID: "191f2736-d3e9-4696-962e-fec0e31de279") : secret "maas-api-e2e-ait-c39c4123-serving-cert" not found
Nov 02 21:36:07.222633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.222612    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.224407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.224377    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.279368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.279338    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt267\" (UniqueName: \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.740471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.740431    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.743068 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.743036    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") pod \"maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:07.989452 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:07.989415    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:08.364437 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:08.364407    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:36:08.365223 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:36:08.365194    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191f2736_d3e9_4696_962e_fec0e31de279.slice/crio-3d6778eb8ce33f5f8466eda7c22293d095608732a4612d13d2be3edd24e508b8 WatchSource:0}: Error finding container 3d6778eb8ce33f5f8466eda7c22293d095608732a4612d13d2be3edd24e508b8: Status 404 returned error can't find the container with id 3d6778eb8ce33f5f8466eda7c22293d095608732a4612d13d2be3edd24e508b8
Nov 02 21:36:09.064225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:09.064181    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" event={"ID":"191f2736-d3e9-4696-962e-fec0e31de279","Type":"ContainerStarted","Data":"3d6778eb8ce33f5f8466eda7c22293d095608732a4612d13d2be3edd24e508b8"}
Nov 02 21:36:10.090518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:10.089647    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" event={"ID":"191f2736-d3e9-4696-962e-fec0e31de279","Type":"ContainerStarted","Data":"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"}
Nov 02 21:36:10.091055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:10.090823    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:10.202490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:10.202435    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" podStartSLOduration=2.258879637 podStartE2EDuration="3.202419703s" totalImagesPullingTime="943.540066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:36:10.199680527 +0000 UTC m=+1399.743472446" watchObservedRunningTime="2026-11-02 21:36:10.202419703 +0000 UTC m=+1399.746211659"
Nov 02 21:36:11.760658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.760618    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:36:11.765247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.765219    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.811070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.811028    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq597\" (UniqueName: \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.811257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.811080    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.811257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.811188    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.811347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.811318    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.825141 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.825103    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:36:11.860878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.860830    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2ccb9fb8-serving-cert\""
Nov 02 21:36:11.912974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.912934    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.913154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.913095    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq597\" (UniqueName: \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.913154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.913127    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.913310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.913167    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.913835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.913802    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.914051 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:11.913968    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-serving-cert: secret "maas-api-e2e-migrate-2ccb9fb8-serving-cert" not found
Nov 02 21:36:11.914807 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:11.914788    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls podName:5c23d50f-735e-4c25-bc1b-c96857a9a6b5 nodeName:}" failed. No retries permitted until 2026-11-02 21:36:12.414031357 +0000 UTC m=+1401.957823272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls") pod "maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" (UID: "5c23d50f-735e-4c25-bc1b-c96857a9a6b5") : secret "maas-api-e2e-migrate-2ccb9fb8-serving-cert" not found
Nov 02 21:36:11.916528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.916502    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:11.967937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:11.967904    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq597\" (UniqueName: \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:12.447622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:12.447584    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:12.450113 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:12.450082    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") pod \"maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:12.679275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:12.679232    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:13.049792 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:36:13.049752    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c23d50f_735e_4c25_bc1b_c96857a9a6b5.slice/crio-0fb0fce96b168f2c4dc356efbd40a6e35015c06f4587d6d656665c42e0d728c5 WatchSource:0}: Error finding container 0fb0fce96b168f2c4dc356efbd40a6e35015c06f4587d6d656665c42e0d728c5: Status 404 returned error can't find the container with id 0fb0fce96b168f2c4dc356efbd40a6e35015c06f4587d6d656665c42e0d728c5
Nov 02 21:36:13.050979 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:13.050950    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:36:13.152574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:13.152535    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" event={"ID":"5c23d50f-735e-4c25-bc1b-c96857a9a6b5","Type":"ContainerStarted","Data":"0fb0fce96b168f2c4dc356efbd40a6e35015c06f4587d6d656665c42e0d728c5"}
Nov 02 21:36:14.162868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:14.162793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" event={"ID":"5c23d50f-735e-4c25-bc1b-c96857a9a6b5","Type":"ContainerStarted","Data":"78c0c262521635a013178d15f9cde022aec997c4da7e4a158fdf681de3e042e4"}
Nov 02 21:36:14.163318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:14.162986    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:14.303201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:14.303089    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" podStartSLOduration=2.371924877 podStartE2EDuration="3.303075063s" totalImagesPullingTime="931.150186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:36:14.299471182 +0000 UTC m=+1403.843263097" watchObservedRunningTime="2026-11-02 21:36:14.303075063 +0000 UTC m=+1403.846866978"
Nov 02 21:36:17.151482 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:17.151451    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:36:20.176927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:20.176895    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:36:22.261470 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:22.261429    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:36:27.021906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.021861    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 21:36:27.027944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.027916    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.069421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.069386    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"]
Nov 02 21:36:27.073717 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.073698    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.077744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.077718    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-445mt\""
Nov 02 21:36:27.118917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.118883    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 21:36:27.119484 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119459    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.119592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119525    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.119685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119661    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.119747 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119698    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb83808-c965-4a5c-bf76-10770370a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.119876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119831    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.119961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119936    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.120009 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.119989    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.120054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120017    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb83808-c965-4a5c-bf76-10770370a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120141    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.120226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120177    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120212    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.120344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120324    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120399 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120378    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgs69\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.120507 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120485    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120561 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120546    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df6m8\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-kube-api-access-df6m8\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120587    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120704 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120678    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.120761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.120727    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.173121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.173079    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"]
Nov 02 21:36:27.221899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.221825    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.221928    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs69\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.221960    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222012    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df6m8\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-kube-api-access-df6m8\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222049    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222081    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222113    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222144    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222246    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222303    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222316    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222339    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb83808-c965-4a5c-bf76-10770370a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222411    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222452    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222484    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222507    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb83808-c965-4a5c-bf76-10770370a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222529    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222558    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222565    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222611    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.222802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222642    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.223151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.222808    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.223151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223014    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.223151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.223151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223107    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.223151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223114    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.223468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223317    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.223578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.223559    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb83808-c965-4a5c-bf76-10770370a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.224622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.224599    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb83808-c965-4a5c-bf76-10770370a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.225071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.225042    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb83808-c965-4a5c-bf76-10770370a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.225257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.225239    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.225341 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.225297    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.278010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.277929    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.278182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.278022    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgs69\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-5dsgz\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.280245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.280215    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.280413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.280384    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df6m8\" (UniqueName: \"kubernetes.io/projected/edb83808-c965-4a5c-bf76-10770370a957-kube-api-access-df6m8\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-x8dsw\" (UID: \"edb83808-c965-4a5c-bf76-10770370a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.343637 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.343605    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:27.389084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.389048    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:27.762051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.762014    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 21:36:27.798389 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:36:27.798348    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e4da4c_9597_4076_9c20_6556a1c6d009.slice/crio-a92d09a0d97ba319f505c41a1613479f6a3d740bcb0d97cb3bd003942c39eaca WatchSource:0}: Error finding container a92d09a0d97ba319f505c41a1613479f6a3d740bcb0d97cb3bd003942c39eaca: Status 404 returned error can't find the container with id a92d09a0d97ba319f505c41a1613479f6a3d740bcb0d97cb3bd003942c39eaca
Nov 02 21:36:27.805320 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.805141    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:27.805448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.805368    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:27.805448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.805414    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:27.809893 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:36:27.809861    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb83808_c965_4a5c_bf76_10770370a957.slice/crio-bb9564635e181e2b3283cacac27f43222b141f7c44aa424d68786cce37a145bb WatchSource:0}: Error finding container bb9564635e181e2b3283cacac27f43222b141f7c44aa424d68786cce37a145bb: Status 404 returned error can't find the container with id bb9564635e181e2b3283cacac27f43222b141f7c44aa424d68786cce37a145bb
Nov 02 21:36:27.811277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.811250    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"]
Nov 02 21:36:27.815994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.815958    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:27.816120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.816041    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:27.816120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:27.816088    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:28.269956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.269914    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" event={"ID":"a3e4da4c-9597-4076-9c20-6556a1c6d009","Type":"ContainerStarted","Data":"48df0f4a6a9151d6a6a93498a78272eb8dc0ca1678033a73b6b66854efd6edfd"}
Nov 02 21:36:28.270666 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.269989    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" event={"ID":"a3e4da4c-9597-4076-9c20-6556a1c6d009","Type":"ContainerStarted","Data":"a92d09a0d97ba319f505c41a1613479f6a3d740bcb0d97cb3bd003942c39eaca"}
Nov 02 21:36:28.272434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.272393    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" event={"ID":"edb83808-c965-4a5c-bf76-10770370a957","Type":"ContainerStarted","Data":"3ea0b6ef372628300f3bc253f5f7de3bff0be447c31f9359fe7b721f78ef0d5f"}
Nov 02 21:36:28.272557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.272466    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" event={"ID":"edb83808-c965-4a5c-bf76-10770370a957","Type":"ContainerStarted","Data":"bb9564635e181e2b3283cacac27f43222b141f7c44aa424d68786cce37a145bb"}
Nov 02 21:36:28.344975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.344937    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:28.345176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.345148    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:28.345245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.345211    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:28.381358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.381300    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podStartSLOduration=2.381286127 podStartE2EDuration="2.381286127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:36:28.380540254 +0000 UTC m=+1417.924332172" watchObservedRunningTime="2026-11-02 21:36:28.381286127 +0000 UTC m=+1417.925078043"
Nov 02 21:36:28.389751 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.389713    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:28.391122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.391100    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:28.391192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.391147    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:28.493929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:28.493868    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podStartSLOduration=2.493834127 podStartE2EDuration="2.493834127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:36:28.490504108 +0000 UTC m=+1418.034296025" watchObservedRunningTime="2026-11-02 21:36:28.493834127 +0000 UTC m=+1418.037626044"
Nov 02 21:36:29.347326 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:29.347284    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:29.347980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:29.347945    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:29.390047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:29.389998    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:29.390323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:29.390285    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:30.349873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:30.346240    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:30.349873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:30.346307    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:30.389948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:30.389914    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:30.390129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:30.389972    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:31.359872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:31.359365    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:31.359872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:31.359434    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:31.389562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:31.389522    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:31.389739 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:31.389581    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:32.350322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:32.350276    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:32.350648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:32.350608    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:32.389419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:32.389386    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:32.389835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:32.389484    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:33.344255 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:33.344212    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:33.344582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:33.344550    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:33.389617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:33.389577    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:33.390035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:33.389637    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:34.344261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:34.344220    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:34.344557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:34.344526    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:34.390223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:34.390190    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:34.390623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:34.390242    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:35.346876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:35.344786    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:35.346876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:35.344868    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:35.393205 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:35.393166    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:36:35.393934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:35.393900    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 21:36:36.344955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:36.344911    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:36:36.345146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:36.344985    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 21:36:37.433046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:37.433002    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:36:37.433683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:37.433646    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:36:38.043630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.043597    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:38.044459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.044433    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:38.046167 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.046132    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 21:36:38.084913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.084830    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:38.085734 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.085709    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:38.088161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:38.088140    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw"
Nov 02 21:36:48.687481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.687445    4888 generic.go:356] "Generic (PLEG): container finished" podID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerID="3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba" exitCode=0
Nov 02 21:36:48.688054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.687528    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerDied","Data":"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"}
Nov 02 21:36:48.688172 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.688154    4888 scope.go:122] "RemoveContainer" containerID="3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"
Nov 02 21:36:48.689897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.689836    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:48.689976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.689954    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:48.690014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:48.689998    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:36:49.697551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:49.697518    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerStarted","Data":"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"}
Nov 02 21:36:49.989284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:49.989191    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:36:49.998429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:49.998383    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:49.998636 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:49.998458    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:50.989613 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:50.989572    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:50.990185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:50.989646    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:51.701237 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:51.701201    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:36:51.989301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:51.989214    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:51.989301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:51.989271    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:52.262526 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:36:52.262437    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:36:52.989297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:52.989261    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:52.989481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:52.989314    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:53.989564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:53.989529    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:53.989987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:53.989595    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:54.989294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:54.989252    4888 patch_prober.go:29] interesting pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:36:54.989520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:54.989322    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 21:36:55.993598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:55.993568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:36:55.994966 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:36:55.994939    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:37:12.376653 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.376615    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:37:12.380766 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.380741    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486367    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486411    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486563    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tsc\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486631    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486801 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486765    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486800    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.486926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486832    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.487048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.486977    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.487048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.487033    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.566430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.566395    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:37:12.588052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588019    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588241 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588161    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tsc\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588268    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588346    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588410 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588377    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588417    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588465    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588572 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588535    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.588626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588568    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.589066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588724    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.589066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.588959    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.589066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.589020    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.589323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.589301    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.589403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.589377    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.590792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.590767    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.591051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.591032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.704808 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.704774    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tsc\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.747906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.739941    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-s27r2\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:12.995589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:12.995495    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:13.419774 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:37:13.419747    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab89e4f_dec4_45c1_baa2_e9943304b23a.slice/crio-7de459061c993cdd12592b9f21f04af61828897501d1f64572f9dc4c0d957aab WatchSource:0}: Error finding container 7de459061c993cdd12592b9f21f04af61828897501d1f64572f9dc4c0d957aab: Status 404 returned error can't find the container with id 7de459061c993cdd12592b9f21f04af61828897501d1f64572f9dc4c0d957aab
Nov 02 21:37:13.420550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.420521    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:37:13.422952 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.422920    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:37:13.423018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.422988    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:37:13.423065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.423021    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:37:13.891692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.891589    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" event={"ID":"5ab89e4f-dec4-45c1-baa2-e9943304b23a","Type":"ContainerStarted","Data":"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"}
Nov 02 21:37:13.891692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.891663    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" event={"ID":"5ab89e4f-dec4-45c1-baa2-e9943304b23a","Type":"ContainerStarted","Data":"7de459061c993cdd12592b9f21f04af61828897501d1f64572f9dc4c0d957aab"}
Nov 02 21:37:13.995622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:13.995579    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:14.997969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:14.997926    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:14.998353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:14.997991    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:15.996064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:15.996026    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:15.996251 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:15.996093    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:16.996596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:16.996561    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:16.997035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:16.996617    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:17.996551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:17.996521    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:17.996734 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:17.996570    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:18.996468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:18.996423    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:18.996666 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:18.996484    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:19.996467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:19.996423    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:19.996890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:19.996534    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:20.996180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:20.996140    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:37:20.996392 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:20.996219    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:37:22.798917 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:37:22.791978    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:37:23.049113 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:23.048990    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:37:23.049550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:23.049507    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:37:24.100021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:24.099979    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:37:24.100805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:24.100743    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:37:25.101983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:25.101944    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:37:25.102588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:25.102557    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:37:26.144043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:26.143995    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:37:26.144592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:26.144072    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:37:26.447401 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:26.447301    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podStartSLOduration=14.447276354 podStartE2EDuration="14.447276354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:37:14.096376563 +0000 UTC m=+1463.640168478" watchObservedRunningTime="2026-11-02 21:37:26.447276354 +0000 UTC m=+1475.991068272"
Nov 02 21:37:26.448576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:26.448550    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:37:27.206031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:27.205982    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:37:27.206631 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:27.206597    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:37:27.551822 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:27.551730    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:27.552246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:27.552219    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:27.553360 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:27.553339    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:37:31.450026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.449912    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" podUID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" containerName="payload-processing" containerID="cri-o://c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3" gracePeriod=25
Nov 02 21:37:31.768760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.768731    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:37:31.786527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.786489    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\") pod \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") "
Nov 02 21:37:31.786725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.786644    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\" (UniqueName: \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\") pod \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\" (UID: \"0a8b18e7-ab31-4f2b-a170-45040d0ef281\") "
Nov 02 21:37:31.787101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.787071    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume" pod "0a8b18e7-ab31-4f2b-a170-45040d0ef281" (UID: "0a8b18e7-ab31-4f2b-a170-45040d0ef281"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:37:31.787534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.787509    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8b18e7-ab31-4f2b-a170-45040d0ef281-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:37:31.789195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.789169    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6" pod "0a8b18e7-ab31-4f2b-a170-45040d0ef281" (UID: "0a8b18e7-ab31-4f2b-a170-45040d0ef281"). InnerVolumeSpecName "kube-api-access-d4qr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:37:31.888999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:31.888964    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4qr6\" (UniqueName: \"kubernetes.io/projected/0a8b18e7-ab31-4f2b-a170-45040d0ef281-kube-api-access-d4qr6\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:37:32.421501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.421464    4888 generic.go:356] "Generic (PLEG): container finished" podID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" containerID="c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3" exitCode=0
Nov 02 21:37:32.421711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.421510    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"
Nov 02 21:37:32.421711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.421514    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" event={"ID":"0a8b18e7-ab31-4f2b-a170-45040d0ef281","Type":"ContainerDied","Data":"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"}
Nov 02 21:37:32.421711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.421652    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg" event={"ID":"0a8b18e7-ab31-4f2b-a170-45040d0ef281","Type":"ContainerDied","Data":"f47678f4964dffdd84003b10068d2faee154cbe1f6f1dd724bcb88335a17b0f8"}
Nov 02 21:37:32.421711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.421690    4888 scope.go:122] "RemoveContainer" containerID="c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"
Nov 02 21:37:32.432455 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.432433    4888 scope.go:122] "RemoveContainer" containerID="c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"
Nov 02 21:37:32.432807 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:37:32.432787    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3\": container with ID starting with c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3 not found: ID does not exist" containerID="c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"
Nov 02 21:37:32.432891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.432816    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3"} err="failed to get container status \"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3\": rpc error: code = NotFound desc = could not find container \"c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3\": container with ID starting with c3f80f841a2ca35711a46cacc58557cf555354f82cd0495641da4ca07ab0e7f3 not found: ID does not exist"
Nov 02 21:37:32.579014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.578980    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:37:32.627480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:32.627446    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg"]
Nov 02 21:37:34.622084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:34.622041    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" path="/var/lib/kubelet/pods/0a8b18e7-ab31-4f2b-a170-45040d0ef281/volumes"
Nov 02 21:37:36.824616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:36.824574    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:37:41.825645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:41.825584    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" podUID="83c02484-883f-41d8-8a6c-84c4994bb95a" containerName="payload-pre-processing" containerID="cri-o://17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66" gracePeriod=25
Nov 02 21:37:42.103115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.103089    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:37:42.139514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.139480    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\") pod \"83c02484-883f-41d8-8a6c-84c4994bb95a\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") "
Nov 02 21:37:42.139703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.139557    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\" (UniqueName: \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\") pod \"83c02484-883f-41d8-8a6c-84c4994bb95a\" (UID: \"83c02484-883f-41d8-8a6c-84c4994bb95a\") "
Nov 02 21:37:42.139978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.139950    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume" pod "83c02484-883f-41d8-8a6c-84c4994bb95a" (UID: "83c02484-883f-41d8-8a6c-84c4994bb95a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:37:42.140300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.140278    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c02484-883f-41d8-8a6c-84c4994bb95a-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:37:42.141732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.141705    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg" pod "83c02484-883f-41d8-8a6c-84c4994bb95a" (UID: "83c02484-883f-41d8-8a6c-84c4994bb95a"). InnerVolumeSpecName "kube-api-access-tsgsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:37:42.242189 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.242147    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsgsg\" (UniqueName: \"kubernetes.io/projected/83c02484-883f-41d8-8a6c-84c4994bb95a-kube-api-access-tsgsg\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:37:42.538960 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.538923    4888 generic.go:356] "Generic (PLEG): container finished" podID="83c02484-883f-41d8-8a6c-84c4994bb95a" containerID="17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66" exitCode=0
Nov 02 21:37:42.539179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.538974    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"
Nov 02 21:37:42.539179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.539013    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" event={"ID":"83c02484-883f-41d8-8a6c-84c4994bb95a","Type":"ContainerDied","Data":"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"}
Nov 02 21:37:42.539179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.539090    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz" event={"ID":"83c02484-883f-41d8-8a6c-84c4994bb95a","Type":"ContainerDied","Data":"4eb38b75fe7d1c0dea71dba0040a967c7f4c7d9450bee852b898cc60895984bb"}
Nov 02 21:37:42.539179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.539112    4888 scope.go:122] "RemoveContainer" containerID="17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"
Nov 02 21:37:42.554594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.554570    4888 scope.go:122] "RemoveContainer" containerID="17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"
Nov 02 21:37:42.555113 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:37:42.555078    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66\": container with ID starting with 17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66 not found: ID does not exist" containerID="17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"
Nov 02 21:37:42.555226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.555123    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66"} err="failed to get container status \"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66\": rpc error: code = NotFound desc = could not find container \"17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66\": container with ID starting with 17a4e6d27c1b3a54ee8936aef2f6026ef9d4751cc5eb31601368d92db9c56d66 not found: ID does not exist"
Nov 02 21:37:42.703045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.703008    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:37:42.748799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:42.748764    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz"]
Nov 02 21:37:44.602004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:44.601968    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83c02484-883f-41d8-8a6c-84c4994bb95a" path="/var/lib/kubelet/pods/83c02484-883f-41d8-8a6c-84c4994bb95a/volumes"
Nov 02 21:37:50.774714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:50.774684    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:37:50.776776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:37:50.776752    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:37:52.260896 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:37:52.260822    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:38:01.006064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.006028    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:38:01.006530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.006236    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" podUID="191f2736-d3e9-4696-962e-fec0e31de279" containerName="maas-api" containerID="cri-o://797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a" gracePeriod=30
Nov 02 21:38:01.641888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.639759    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:38:01.823999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.822988    4888 generic.go:356] "Generic (PLEG): container finished" podID="191f2736-d3e9-4696-962e-fec0e31de279" containerID="797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a" exitCode=0
Nov 02 21:38:01.823999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.823248    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" event={"ID":"191f2736-d3e9-4696-962e-fec0e31de279","Type":"ContainerDied","Data":"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"}
Nov 02 21:38:01.823999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.823311    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj" event={"ID":"191f2736-d3e9-4696-962e-fec0e31de279","Type":"ContainerDied","Data":"3d6778eb8ce33f5f8466eda7c22293d095608732a4612d13d2be3edd24e508b8"}
Nov 02 21:38:01.823999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.823348    4888 scope.go:122] "RemoveContainer" containerID="797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"
Nov 02 21:38:01.823999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.823522    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"
Nov 02 21:38:01.857974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.841523    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") pod \"191f2736-d3e9-4696-962e-fec0e31de279\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") "
Nov 02 21:38:01.866189 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.866155    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\") pod \"191f2736-d3e9-4696-962e-fec0e31de279\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") "
Nov 02 21:38:01.867036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.867012    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\") pod \"191f2736-d3e9-4696-962e-fec0e31de279\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") "
Nov 02 21:38:01.869664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.869638    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\" (UniqueName: \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\") pod \"191f2736-d3e9-4696-962e-fec0e31de279\" (UID: \"191f2736-d3e9-4696-962e-fec0e31de279\") "
Nov 02 21:38:01.871056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.868221    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca" pod "191f2736-d3e9-4696-962e-fec0e31de279" (UID: "191f2736-d3e9-4696-962e-fec0e31de279"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:38:01.871056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.871022    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191f2736-d3e9-4696-962e-fec0e31de279-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:01.876146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.876095    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls" pod "191f2736-d3e9-4696-962e-fec0e31de279" (UID: "191f2736-d3e9-4696-962e-fec0e31de279"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:38:01.876276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.876136    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs" pod "191f2736-d3e9-4696-962e-fec0e31de279" (UID: "191f2736-d3e9-4696-962e-fec0e31de279"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:38:01.876834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.876465    4888 scope.go:122] "RemoveContainer" containerID="797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"
Nov 02 21:38:01.876834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.876811    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267" pod "191f2736-d3e9-4696-962e-fec0e31de279" (UID: "191f2736-d3e9-4696-962e-fec0e31de279"). InnerVolumeSpecName "kube-api-access-rt267". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:38:01.877042 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:38:01.876918    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a\": container with ID starting with 797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a not found: ID does not exist" containerID="797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"
Nov 02 21:38:01.877042 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.876958    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a"} err="failed to get container status \"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a\": rpc error: code = NotFound desc = could not find container \"797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a\": container with ID starting with 797d8c90bfd62c166186f8c08bc278f785459b090f55c4204c9b74156aeba34a not found: ID does not exist"
Nov 02 21:38:01.983887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.982392    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt267\" (UniqueName: \"kubernetes.io/projected/191f2736-d3e9-4696-962e-fec0e31de279-kube-api-access-rt267\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:01.983887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.982438    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:01.983887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:01.982453    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191f2736-d3e9-4696-962e-fec0e31de279-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:02.173941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.173901    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:38:02.187221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.174677    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" podUID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" containerName="maas-api" containerID="cri-o://78c0c262521635a013178d15f9cde022aec997c4da7e4a158fdf681de3e042e4" gracePeriod=30
Nov 02 21:38:02.308028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.307903    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:38:02.317033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.316998    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c39c4123-54d56b8d79-h8sgj"]
Nov 02 21:38:02.844928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.842944    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="191f2736-d3e9-4696-962e-fec0e31de279" path="/var/lib/kubelet/pods/191f2736-d3e9-4696-962e-fec0e31de279/volumes"
Nov 02 21:38:02.851725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.851698    4888 generic.go:356] "Generic (PLEG): container finished" podID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" containerID="78c0c262521635a013178d15f9cde022aec997c4da7e4a158fdf681de3e042e4" exitCode=0
Nov 02 21:38:02.851839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.851817    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" event={"ID":"5c23d50f-735e-4c25-bc1b-c96857a9a6b5","Type":"ContainerDied","Data":"78c0c262521635a013178d15f9cde022aec997c4da7e4a158fdf681de3e042e4"}
Nov 02 21:38:02.881395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:02.881368    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:38:03.046906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.046823    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\" (UniqueName: \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\") pod \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") "
Nov 02 21:38:03.047117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.046966    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\") pod \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") "
Nov 02 21:38:03.047187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.047127    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") pod \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") "
Nov 02 21:38:03.047250 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.047240    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\") pod \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\" (UID: \"5c23d50f-735e-4c25-bc1b-c96857a9a6b5\") "
Nov 02 21:38:03.047343 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.047314    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca" pod "5c23d50f-735e-4c25-bc1b-c96857a9a6b5" (UID: "5c23d50f-735e-4c25-bc1b-c96857a9a6b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:38:03.047705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.047682    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:03.049190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.049157    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597" pod "5c23d50f-735e-4c25-bc1b-c96857a9a6b5" (UID: "5c23d50f-735e-4c25-bc1b-c96857a9a6b5"). InnerVolumeSpecName "kube-api-access-hq597". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:38:03.049369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.049345    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs" pod "5c23d50f-735e-4c25-bc1b-c96857a9a6b5" (UID: "5c23d50f-735e-4c25-bc1b-c96857a9a6b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:38:03.049442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.049358    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls" pod "5c23d50f-735e-4c25-bc1b-c96857a9a6b5" (UID: "5c23d50f-735e-4c25-bc1b-c96857a9a6b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:38:03.162877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.155687    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:03.162877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.155724    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq597\" (UniqueName: \"kubernetes.io/projected/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-kube-api-access-hq597\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:03.162877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.155739    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c23d50f-735e-4c25-bc1b-c96857a9a6b5-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:03.402474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.402435    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:38:03.402975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.402723    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy" containerID="cri-o://70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf" gracePeriod=30
Nov 02 21:38:03.870127 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.870090    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg" event={"ID":"5c23d50f-735e-4c25-bc1b-c96857a9a6b5","Type":"ContainerDied","Data":"0fb0fce96b168f2c4dc356efbd40a6e35015c06f4587d6d656665c42e0d728c5"}
Nov 02 21:38:03.870308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.870149    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"
Nov 02 21:38:03.870308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:03.870208    4888 scope.go:122] "RemoveContainer" containerID="78c0c262521635a013178d15f9cde022aec997c4da7e4a158fdf681de3e042e4"
Nov 02 21:38:04.028226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:04.028195    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:38:04.032080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:04.032045    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2ccb9fb8-66c69c69b9-fqgqg"]
Nov 02 21:38:04.666619 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:04.666578    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" path="/var/lib/kubelet/pods/5c23d50f-735e-4c25-bc1b-c96857a9a6b5/volumes"
Nov 02 21:38:05.011350 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011268    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:38:05.011959 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011942    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" containerName="maas-api"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011959    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" containerName="maas-api"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011979    4888 cpu_manager.go:414] "containerMap: removing container" podUID="83c02484-883f-41d8-8a6c-84c4994bb95a" containerName="payload-pre-processing"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011984    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83c02484-883f-41d8-8a6c-84c4994bb95a" containerName="payload-pre-processing"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.011997    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" containerName="payload-processing"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012002    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" containerName="payload-processing"
Nov 02 21:38:05.012014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012016    4888 cpu_manager.go:414] "containerMap: removing container" podUID="191f2736-d3e9-4696-962e-fec0e31de279" containerName="maas-api"
Nov 02 21:38:05.012248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012021    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191f2736-d3e9-4696-962e-fec0e31de279" containerName="maas-api"
Nov 02 21:38:05.012248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012155    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="83c02484-883f-41d8-8a6c-84c4994bb95a" containerName="payload-pre-processing"
Nov 02 21:38:05.012248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012172    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a8b18e7-ab31-4f2b-a170-45040d0ef281" containerName="payload-processing"
Nov 02 21:38:05.012248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012182    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="191f2736-d3e9-4696-962e-fec0e31de279" containerName="maas-api"
Nov 02 21:38:05.012248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.012193    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c23d50f-735e-4c25-bc1b-c96857a9a6b5" containerName="maas-api"
Nov 02 21:38:05.017208 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.017185    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.130159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.118106    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5935b530-openshift-default-dockercfg-zhjrw\""
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.136887    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.136951    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.136988    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137040    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr5rt\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137091    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137148    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137214    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137269    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.143870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.137416    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.161592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.151208    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246003    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246151    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246191    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246219    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246268    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr5rt\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246313    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246361    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246419    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.252877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.246469    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.285934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.285373    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.285934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.285711    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.304868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.286590    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.304868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.301703    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.304868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.302328    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.313901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.312244    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.430261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.430191    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.430974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.430793    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr5rt\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.440155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.434378    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\") pod \"e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") " pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:05.638953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:05.636083    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:06.468551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.468514    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:38:06.469010 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:38:06.468988    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fc9342_0659_42f5_b0ed_5475149426da.slice/crio-e395366c7701980a9177010fccaadbd83a5a7849f7c50ae221d9e6dba3d2283e WatchSource:0}: Error finding container e395366c7701980a9177010fccaadbd83a5a7849f7c50ae221d9e6dba3d2283e: Status 404 returned error can't find the container with id e395366c7701980a9177010fccaadbd83a5a7849f7c50ae221d9e6dba3d2283e
Nov 02 21:38:06.472217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.472178    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:06.472331 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.472266    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:06.472331 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.472314    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:06.987406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.987367    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerStarted","Data":"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"}
Nov 02 21:38:06.987620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:06.987422    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerStarted","Data":"e395366c7701980a9177010fccaadbd83a5a7849f7c50ae221d9e6dba3d2283e"}
Nov 02 21:38:07.146042 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.145974    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podStartSLOduration=3.145951982 podStartE2EDuration="3.145951982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:07.145250362 +0000 UTC m=+1516.689042280" watchObservedRunningTime="2026-11-02 21:38:07.145951982 +0000 UTC m=+1516.689743901"
Nov 02 21:38:07.241308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.241215    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:38:07.245798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.245772    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.299882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.299048    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e942516f-openshift-default-dockercfg-mf9mq\""
Nov 02 21:38:07.363098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.363059    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.363436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.363411    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.363594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.363578    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.363728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.363711    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.363958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.363926    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsl6c\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.364118 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.364093    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.364253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.364237    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.364594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.364575    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.364792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.364778    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.471941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.470930    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsl6c\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.476483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.471001    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.476483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.472570    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.476483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.472788    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.476483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.472867    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.479105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.477630    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.479105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.477743    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.479105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.477788    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.479105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.477822    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.480242 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.480214    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.480645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.480625    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.481705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.481444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.483049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.482958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.483708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.483656    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.483872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.483838    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:38:07.489000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.488946    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.491965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.491834    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.611310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.611276    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.611573 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.611556    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsl6c\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\") pod \"e2e-ait-e942516f-openshift-default-7df544c549-lq6lh\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") " pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:07.685240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.678699    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:07.700297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.700244    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:07.863922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:07.863000    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:08.000658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.000626    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:08.003989 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.003655    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:38:08.338742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.338710    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:38:08.338998 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:38:08.338973    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc148f0bf_fd9c_4803_8c1d_69f7f281c323.slice/crio-211510ad2be1ad4f18d9ed0accf4dab699d92ae24785017804acf840203306be WatchSource:0}: Error finding container 211510ad2be1ad4f18d9ed0accf4dab699d92ae24785017804acf840203306be: Status 404 returned error can't find the container with id 211510ad2be1ad4f18d9ed0accf4dab699d92ae24785017804acf840203306be
Nov 02 21:38:08.342578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.342539    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:08.342675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.342635    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:08.342713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.342681    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:38:08.751467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.751434    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:38:08.830084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830038    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.830285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830101    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.830285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830138    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.830285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830168    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.830973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830646    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:38:08.830973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830657    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.830973 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.830758    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.831246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.831066    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:08.831402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.831372    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.831478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.831461    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.831601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.831582    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\") pod \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\" (UID: \"1ecc7e64-b1b1-4356-bbbf-007f6348e168\") "
Nov 02 21:38:08.832038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.831925    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:08.832038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832013    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:08.832388 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832366    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:08.832499 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832464    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.832643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832509    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.832643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832529    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.832643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832545    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.832810 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.832777    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:38:08.833107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.833083    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:08.833324 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.833304    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:38:08.833989 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.833962    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn" pod "1ecc7e64-b1b1-4356-bbbf-007f6348e168" (UID: "1ecc7e64-b1b1-4356-bbbf-007f6348e168"). InnerVolumeSpecName "kube-api-access-nppnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:38:08.934598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.934560    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.934598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.934598    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.934814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.934611    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.934814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.934623    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ecc7e64-b1b1-4356-bbbf-007f6348e168-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:08.934814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:08.934636    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nppnn\" (UniqueName: \"kubernetes.io/projected/1ecc7e64-b1b1-4356-bbbf-007f6348e168-kube-api-access-nppnn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:38:09.008390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.008358    4888 generic.go:356] "Generic (PLEG): container finished" podID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerID="70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf" exitCode=0
Nov 02 21:38:09.008569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.008425    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"
Nov 02 21:38:09.008569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.008446    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerDied","Data":"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"}
Nov 02 21:38:09.008569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.008531    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh" event={"ID":"1ecc7e64-b1b1-4356-bbbf-007f6348e168","Type":"ContainerDied","Data":"f6de2392059d15a587424a3e4ab2e352720cd17fe11c7a456d119f9f31fbe272"}
Nov 02 21:38:09.008742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.008565    4888 scope.go:122] "RemoveContainer" containerID="70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"
Nov 02 21:38:09.010639 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.010613    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerStarted","Data":"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"}
Nov 02 21:38:09.010737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.010663    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerStarted","Data":"211510ad2be1ad4f18d9ed0accf4dab699d92ae24785017804acf840203306be"}
Nov 02 21:38:09.020509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.020482    4888 scope.go:122] "RemoveContainer" containerID="3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"
Nov 02 21:38:09.031045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.031015    4888 scope.go:122] "RemoveContainer" containerID="70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"
Nov 02 21:38:09.031361 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:38:09.031340    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf\": container with ID starting with 70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf not found: ID does not exist" containerID="70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"
Nov 02 21:38:09.031433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.031370    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf"} err="failed to get container status \"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf\": rpc error: code = NotFound desc = could not find container \"70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf\": container with ID starting with 70773966a1034be7bee78fdb6368de496a9fd7a86ff0b7ba1922bdf16038b2bf not found: ID does not exist"
Nov 02 21:38:09.031433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.031387    4888 scope.go:122] "RemoveContainer" containerID="3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"
Nov 02 21:38:09.031673 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:38:09.031652    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba\": container with ID starting with 3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba not found: ID does not exist" containerID="3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"
Nov 02 21:38:09.031719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.031682    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba"} err="failed to get container status \"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba\": rpc error: code = NotFound desc = could not find container \"3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba\": container with ID starting with 3367553b91d87aea611f52545168e5d7f00f5fdf6f42c3a151047ad871f953ba not found: ID does not exist"
Nov 02 21:38:09.175003 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.174933    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podStartSLOduration=2.174911419 podStartE2EDuration="2.174911419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:09.170944967 +0000 UTC m=+1518.714736885" watchObservedRunningTime="2026-11-02 21:38:09.174911419 +0000 UTC m=+1518.718703335"
Nov 02 21:38:09.288439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.287425    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:38:09.293567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.293538    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh"]
Nov 02 21:38:09.879050 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.878917    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:09.992871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:09.991240    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:10.077416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:10.077367    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:10.104262 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:10.104231    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:38:10.634296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:10.628322    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" path="/var/lib/kubelet/pods/1ecc7e64-b1b1-4356-bbbf-007f6348e168/volumes"
Nov 02 21:38:12.914881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:12.908139    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018557    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018588    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018608    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018616    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018803    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.022967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.018816    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ecc7e64-b1b1-4356-bbbf-007f6348e168" containerName="istio-proxy"
Nov 02 21:38:13.089870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.089753    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:38:13.090042 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.089898    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.105422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.093341    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:38:13.105828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.105804    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.122908 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.112327    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:38:13.143505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.143475    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5935b530\""
Nov 02 21:38:13.144535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.143843    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5935b530-dockercfg-w5jp9\""
Nov 02 21:38:13.234479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.215272    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k654s\" (UniqueName: \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.234479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.215460    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.352750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.335008    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.352750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.335068    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5b29\" (UniqueName: \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.352750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.335125    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k654s\" (UniqueName: \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.352750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.335402    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.354401 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.353594    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.410465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.410433    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k654s\" (UniqueName: \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\") pod \"payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.436934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.436897    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.436934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.436939    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5b29\" (UniqueName: \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.437666 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.437647    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.491620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.491540    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5b29\" (UniqueName: \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\") pod \"payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") " pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:13.668545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.668510    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:13.683386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:13.683356    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:14.045056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.045016    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:38:14.093060 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:38:14.093022    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15bfba6_e818_4ca7_be86_c7e231166888.slice/crio-2c0fd9e751ceeeab006a1d6069af79685673802a3c526c894936b9d887dd4e4b WatchSource:0}: Error finding container 2c0fd9e751ceeeab006a1d6069af79685673802a3c526c894936b9d887dd4e4b: Status 404 returned error can't find the container with id 2c0fd9e751ceeeab006a1d6069af79685673802a3c526c894936b9d887dd4e4b
Nov 02 21:38:14.096179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.096153    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:38:14.197970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.197926    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" event={"ID":"68e0d317-90d6-4e68-8f4b-71185d7047da","Type":"ContainerStarted","Data":"fe595302edd8371694ba4e6e31e1dfce25c36dcf7b14bdede9f80240473189e7"}
Nov 02 21:38:14.198143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.197992    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" event={"ID":"68e0d317-90d6-4e68-8f4b-71185d7047da","Type":"ContainerStarted","Data":"1d63197b9c32afd664571b72e0ca7a0b44dba5d7b728d69c25382a05dbc1b17c"}
Nov 02 21:38:14.198143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.198031    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:14.199774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.199748    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" event={"ID":"c15bfba6-e818-4ca7-be86-c7e231166888","Type":"ContainerStarted","Data":"6e86f5f13637d922e4c9cec9bd34185d1658163dcf0b97ec0acbbf13fb453ca7"}
Nov 02 21:38:14.199939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.199799    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" event={"ID":"c15bfba6-e818-4ca7-be86-c7e231166888","Type":"ContainerStarted","Data":"2c0fd9e751ceeeab006a1d6069af79685673802a3c526c894936b9d887dd4e4b"}
Nov 02 21:38:14.199939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.199821    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:14.311072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.310971    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" podStartSLOduration=2.310953654 podStartE2EDuration="2.310953654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:14.3104472 +0000 UTC m=+1523.854239118" watchObservedRunningTime="2026-11-02 21:38:14.310953654 +0000 UTC m=+1523.854745571"
Nov 02 21:38:14.478505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:14.478438    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" podStartSLOduration=2.478415763 podStartE2EDuration="2.478415763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:14.470719153 +0000 UTC m=+1524.014511069" watchObservedRunningTime="2026-11-02 21:38:14.478415763 +0000 UTC m=+1524.022207680"
Nov 02 21:38:17.667707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.667403    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:38:17.704870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.701360    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:38:17.704870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.701479    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:17.797205 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.797151    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k87q8\" (UniqueName: \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:17.821290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.821255    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:17.821571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.814688    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e942516f-dockercfg-xl4jk\""
Nov 02 21:38:17.821682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.814771    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e942516f\""
Nov 02 21:38:17.821682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.814568    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:38:17.960702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.875492    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:17.960702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.886560    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:38:17.960702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.956615    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87q8\" (UniqueName: \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:17.960702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.956808    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:17.960702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:17.960414    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:18.019887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.018736    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87q8\" (UniqueName: \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\") pod \"payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:18.065017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.064867    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.065667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.065536    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjvw4\" (UniqueName: \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.168907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.167943    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.168907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.168037    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjvw4\" (UniqueName: \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.172888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.170033    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.180604 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.178325    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:18.249946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.249328    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjvw4\" (UniqueName: \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\") pod \"payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") " pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.296209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.296117    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:18.617962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.617929    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:38:18.714224 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:38:18.714193    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40a058a_0816_4667_940b_63d4fc182573.slice/crio-35dc7c0d85bd71f2cd0871e6c9c36cb7c9890eade185e2194b3066af0d931e07 WatchSource:0}: Error finding container 35dc7c0d85bd71f2cd0871e6c9c36cb7c9890eade185e2194b3066af0d931e07: Status 404 returned error can't find the container with id 35dc7c0d85bd71f2cd0871e6c9c36cb7c9890eade185e2194b3066af0d931e07
Nov 02 21:38:18.715035 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:18.714901    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:38:19.265195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.265151    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" event={"ID":"e40a058a-0816-4667-940b-63d4fc182573","Type":"ContainerStarted","Data":"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"}
Nov 02 21:38:19.265375 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.265218    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" event={"ID":"e40a058a-0816-4667-940b-63d4fc182573","Type":"ContainerStarted","Data":"35dc7c0d85bd71f2cd0871e6c9c36cb7c9890eade185e2194b3066af0d931e07"}
Nov 02 21:38:19.265375 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.265352    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:19.267316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.267283    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" event={"ID":"63837466-414b-43b3-9904-b1c96ebaf755","Type":"ContainerStarted","Data":"1e4781a3de2ce4a26c44d7d4aab4a8fd810a4cff86f5049f74d06e9481885a68"}
Nov 02 21:38:19.267439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.267338    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:19.267439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.267360    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" event={"ID":"63837466-414b-43b3-9904-b1c96ebaf755","Type":"ContainerStarted","Data":"95ae146dfde95a2df2b6bd81f56ffb417f58c7d064c475181b2549186f10e7f7"}
Nov 02 21:38:19.377209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.377155    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" podStartSLOduration=2.377138557 podStartE2EDuration="2.377138557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:19.373509647 +0000 UTC m=+1528.917301564" watchObservedRunningTime="2026-11-02 21:38:19.377138557 +0000 UTC m=+1528.920930503"
Nov 02 21:38:19.511876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:19.510215    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" podStartSLOduration=2.510191854 podStartE2EDuration="2.510191854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:19.498072122 +0000 UTC m=+1529.041864051" watchObservedRunningTime="2026-11-02 21:38:19.510191854 +0000 UTC m=+1529.053983771"
Nov 02 21:38:22.263605 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:38:22.263559    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:38:25.209335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:25.209303    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:38:25.209745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:25.209402    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:38:30.276797 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:30.276767    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:38:30.277386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:30.276834    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:38:52.260293 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:38:52.260253    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:38:53.053936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.053902    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:38:53.059458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.059432    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.110447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.110412    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:38:53.153614 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.153569    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5935b530-serving-cert\""
Nov 02 21:38:53.257838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.257801    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.258033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.257841    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.258033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.257974    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.258253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.258227    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8tr4\" (UniqueName: \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.361398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.361300    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8tr4\" (UniqueName: \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.362057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.362031    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.362193 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.362178    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.362315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.362301    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.364176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.363163    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.374247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.374188    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.392728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.392699    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.429311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.427654    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8tr4\" (UniqueName: \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\") pod \"maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:53.674193 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:53.674153    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:54.047489 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:38:54.047457    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb147d9c_8b20_45df_890f_ccc3468ac5a3.slice/crio-5774cf4ac8506ec3aca96f0322981bb604e626e86ead0c14203b6be6f3ce774c WatchSource:0}: Error finding container 5774cf4ac8506ec3aca96f0322981bb604e626e86ead0c14203b6be6f3ce774c: Status 404 returned error can't find the container with id 5774cf4ac8506ec3aca96f0322981bb604e626e86ead0c14203b6be6f3ce774c
Nov 02 21:38:54.048891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:54.048866    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:38:54.553865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:54.553811    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" event={"ID":"bb147d9c-8b20-45df-890f-ccc3468ac5a3","Type":"ContainerStarted","Data":"5774cf4ac8506ec3aca96f0322981bb604e626e86ead0c14203b6be6f3ce774c"}
Nov 02 21:38:55.562439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:55.562402    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" event={"ID":"bb147d9c-8b20-45df-890f-ccc3468ac5a3","Type":"ContainerStarted","Data":"0093dba5bdcae5cde6d18d23be0902c3b84f2ecc368c7a8f695423e036c75474"}
Nov 02 21:38:55.562839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:55.562468    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:38:55.673970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:38:55.673919    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" podStartSLOduration=1.692688322 podStartE2EDuration="2.673902147s" totalImagesPullingTime="981.213825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:55.673044116 +0000 UTC m=+1565.216836032" watchObservedRunningTime="2026-11-02 21:38:55.673902147 +0000 UTC m=+1565.217694062"
Nov 02 21:39:01.576460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:01.576382    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:39:04.666612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:04.666574    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:39:04.667063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:04.666812    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy" containerID="cri-o://313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2" gracePeriod=30
Nov 02 21:39:09.945174 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:09.945104    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:39:10.039039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039004    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039074    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039149    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039206    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039281    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039316    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:39:10.039387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039347    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039542 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039470    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039542 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039519    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039582    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\") pod \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\" (UID: \"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5\") "
Nov 02 21:39:10.039665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039581    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:39:10.040361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039867    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:39:10.040361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.039948    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:39:10.040361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040039    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:39:10.040593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040432    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.040593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040455    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.040593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040471    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.040593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040486    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.040593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.040500    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.041686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.041662    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:39:10.041798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.041760    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "kube-api-access-nsnz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:39:10.041932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.041915    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:39:10.042002 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.041983    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo" pod "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" (UID: "6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:39:10.141618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.141583    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsnz8\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-kube-api-access-nsnz8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.141618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.141616    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.141829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.141629    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.141829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.141638    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:10.766058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.766024    4888 generic.go:356] "Generic (PLEG): container finished" podID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerID="313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2" exitCode=0
Nov 02 21:39:10.766231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.766078    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"
Nov 02 21:39:10.766231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.766107    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" event={"ID":"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5","Type":"ContainerDied","Data":"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"}
Nov 02 21:39:10.766231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.766158    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j" event={"ID":"6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5","Type":"ContainerDied","Data":"cde785da491b1790fe73b085a37b985e51250f316a65d221df9ad81a62d6882b"}
Nov 02 21:39:10.766231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.766188    4888 scope.go:122] "RemoveContainer" containerID="313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"
Nov 02 21:39:10.776501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.776476    4888 scope.go:122] "RemoveContainer" containerID="313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"
Nov 02 21:39:10.776802 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:39:10.776781    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2\": container with ID starting with 313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2 not found: ID does not exist" containerID="313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"
Nov 02 21:39:10.776896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.776809    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2"} err="failed to get container status \"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2\": rpc error: code = NotFound desc = could not find container \"313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2\": container with ID starting with 313d05d84c9432690c3bd0ac78021c38b3aeb872b843ebed9815c555fa716ce2 not found: ID does not exist"
Nov 02 21:39:10.971465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:10.971431    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:39:11.033733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:11.033695    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j"]
Nov 02 21:39:12.601540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:12.601508    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" path="/var/lib/kubelet/pods/6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5/volumes"
Nov 02 21:39:22.296953 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:39:22.296902    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:39:24.939137 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:24.939098    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:39:24.959958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:24.959922    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy"
Nov 02 21:39:24.960150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:24.960134    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy"
Nov 02 21:39:24.975952 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:24.960728    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b3c7cb9-23a6-49a5-ac4f-e633f02e1bd5" containerName="istio-proxy"
Nov 02 21:39:25.296922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.290656    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:39:25.296922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.290791    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.398646 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.398605    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6b983e02-openshift-default-dockercfg-xbqwp\""
Nov 02 21:39:25.437453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.437414    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrmk\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.437744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.437716    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.438077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.437998    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.438261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.438234    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.438411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.438395    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.438649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.438631    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.438791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.438777    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.447025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.446994    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.447305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.447287    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586341    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586490    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586643    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrmk\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586682    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586815    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586868    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.586904    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.587065    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.594439    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.596039    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.625402    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.630087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.627185    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:39:25.653492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.643867    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.653492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.644127    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.806879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.798904    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.806879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.800003    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:39:25.850876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.840837    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 02 21:39:25.850876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.845473    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrmk\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.929804 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.929732    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:25.933154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:25.933106    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:26.005551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:26.005513    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:26.677671 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:26.677630    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:39:26.710103 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:39:26.710055    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36e86b4_ae71_4466_82c8_6412967f2836.slice/crio-13183a2ad3b3f54915a9b966fba822353b5d62e619dc78d369adc4503d8eb8c0 WatchSource:0}: Error finding container 13183a2ad3b3f54915a9b966fba822353b5d62e619dc78d369adc4503d8eb8c0: Status 404 returned error can't find the container with id 13183a2ad3b3f54915a9b966fba822353b5d62e619dc78d369adc4503d8eb8c0
Nov 02 21:39:26.728117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:26.719683    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:26.728117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:26.719797    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:26.728117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:26.719844    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:27.367915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.367878    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:39:27.414991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.414950    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerStarted","Data":"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"}
Nov 02 21:39:27.415258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.415240    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerStarted","Data":"13183a2ad3b3f54915a9b966fba822353b5d62e619dc78d369adc4503d8eb8c0"}
Nov 02 21:39:27.546775 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.546738    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 21:39:27.552493 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.552464    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.594098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.594053    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podStartSLOduration=3.594037454 podStartE2EDuration="3.594037454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:27.593545918 +0000 UTC m=+1597.137337834" watchObservedRunningTime="2026-11-02 21:39:27.594037454 +0000 UTC m=+1597.137829370"
Nov 02 21:39:27.657066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.637438    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 21:39:27.657066 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.653323    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-f4a242-openshift-default-dockercfg-cjtnm\""
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.662686    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.662738    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.662797    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.662916    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.662947    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.663018    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.663093    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qng7r\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.663170    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.663873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.663286    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.765376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.765340    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.765607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.765587    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766074    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766306    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766446 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766431    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766604 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766588    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766750    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qng7r\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.766958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766941    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.767183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.767161    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.768203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.768181    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.768601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.768584    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.768708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.766128    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.768926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.768899    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.769187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.769160    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.769779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.769756    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.773593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.773571    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.902959 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.902230    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qng7r\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:27.919145 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:27.919055    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\") pod \"e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") " pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:28.011947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.011901    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:28.052221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.052189    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:28.207032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.202175    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:28.478523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.478410    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:28.480342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.480172    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:39:28.991900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.987083    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:28.991900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.987187    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:28.991900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.987229    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:28.991900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:28.991486    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 21:39:29.496990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:29.496902    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerStarted","Data":"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"}
Nov 02 21:39:29.496990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:29.496967    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerStarted","Data":"d2549a459193e90faf262c24e0d7be3bcdb4c3e1f1ed48934232428c2abcf4a1"}
Nov 02 21:39:29.611014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:29.610939    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podStartSLOduration=2.610924748 podStartE2EDuration="2.610924748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:29.607564612 +0000 UTC m=+1599.151356529" watchObservedRunningTime="2026-11-02 21:39:29.610924748 +0000 UTC m=+1599.154716664"
Nov 02 21:39:30.207633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:30.207596    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:30.215163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:30.215131    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:30.503540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:30.503415    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:30.505426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:30.505400    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:39:32.369039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.368987    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" podUID="2d61f536-8217-413b-a18b-9db80ca82c32" containerName="payload-pre-processing" containerID="cri-o://8735cc8d63427c364fc8191dc19e0c0862e14df509a8fce76fd145dfe2349171" gracePeriod=25
Nov 02 21:39:32.523756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.523364    4888 generic.go:356] "Generic (PLEG): container finished" podID="2d61f536-8217-413b-a18b-9db80ca82c32" containerID="8735cc8d63427c364fc8191dc19e0c0862e14df509a8fce76fd145dfe2349171" exitCode=0
Nov 02 21:39:32.524912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.523951    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" event={"ID":"2d61f536-8217-413b-a18b-9db80ca82c32","Type":"ContainerDied","Data":"8735cc8d63427c364fc8191dc19e0c0862e14df509a8fce76fd145dfe2349171"}
Nov 02 21:39:32.733496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.733461    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:39:32.860412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.860378    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\" (UniqueName: \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\") pod \"2d61f536-8217-413b-a18b-9db80ca82c32\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") "
Nov 02 21:39:32.860610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.860507    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\") pod \"2d61f536-8217-413b-a18b-9db80ca82c32\" (UID: \"2d61f536-8217-413b-a18b-9db80ca82c32\") "
Nov 02 21:39:32.860981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.860953    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume" pod "2d61f536-8217-413b-a18b-9db80ca82c32" (UID: "2d61f536-8217-413b-a18b-9db80ca82c32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:39:32.861304 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.861281    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d61f536-8217-413b-a18b-9db80ca82c32-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:32.869496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.863980    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j" pod "2d61f536-8217-413b-a18b-9db80ca82c32" (UID: "2d61f536-8217-413b-a18b-9db80ca82c32"). InnerVolumeSpecName "kube-api-access-9s69j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:39:32.962901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:32.962834    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s69j\" (UniqueName: \"kubernetes.io/projected/2d61f536-8217-413b-a18b-9db80ca82c32-kube-api-access-9s69j\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:39:33.533504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.533471    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8" event={"ID":"2d61f536-8217-413b-a18b-9db80ca82c32","Type":"ContainerDied","Data":"c84abafa7f831895f79a38188f2992bf328077e39399e5f9385be64c12e72495"}
Nov 02 21:39:33.536406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.533547    4888 scope.go:122] "RemoveContainer" containerID="8735cc8d63427c364fc8191dc19e0c0862e14df509a8fce76fd145dfe2349171"
Nov 02 21:39:33.536406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.533482    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"
Nov 02 21:39:33.536406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.535820    4888 generic.go:356] "Generic (PLEG): container finished" podID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerID="08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af" exitCode=0
Nov 02 21:39:33.536406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.535879    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerDied","Data":"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"}
Nov 02 21:39:33.536406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.536396    4888 scope.go:122] "RemoveContainer" containerID="08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"
Nov 02 21:39:33.538336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.538299    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:33.538468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.538398    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:33.538468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.538452    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:33.832368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.832259    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:39:33.837830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:33.837799    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8"]
Nov 02 21:39:34.610353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.608534    4888 generic.go:356] "Generic (PLEG): container finished" podID="48fc9342-0659-42f5-b0ed-5475149426da" containerID="3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0" exitCode=0
Nov 02 21:39:34.617951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.617915    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d61f536-8217-413b-a18b-9db80ca82c32" path="/var/lib/kubelet/pods/2d61f536-8217-413b-a18b-9db80ca82c32/volumes"
Nov 02 21:39:34.618669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.618636    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerDied","Data":"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"}
Nov 02 21:39:34.618886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.618841    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerStarted","Data":"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"}
Nov 02 21:39:34.619549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.619498    4888 scope.go:122] "RemoveContainer" containerID="3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"
Nov 02 21:39:34.622513 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.622434    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:34.623911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.623886    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:34.624104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.624081    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:39:34.866936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.866145    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:39:34.874938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.871659    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:34.874938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:34.871715    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:35.702883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:35.695268    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerStarted","Data":"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"}
Nov 02 21:39:35.902903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:35.874474    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:35.902903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:35.874568    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:36.640117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:36.639149    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:39:36.651918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:36.645969    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:36.651918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:36.646050    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:36.865267 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:36.865224    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:36.865695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:36.865303    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:37.641873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:37.641274    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:37.641873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:37.641342    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:37.864529 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:37.864490    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:37.864718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:37.864573    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:38.001151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:38.001114    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:39:38.797879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:38.794688    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:38.797879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:38.794747    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:38.883871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:38.867760    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:38.884203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:38.884164    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:39.641753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:39.641714    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:39.643225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:39.643186    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:39.865440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:39.865265    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:39.865440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:39.865347    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:40.077209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:40.077165    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:39:40.646337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:40.646299    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:40.646946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:40.646908    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:40.865873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:40.865731    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 21:39:40.865873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:40.865790    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 21:39:41.639294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:41.639254    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 21:39:41.639465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:41.639310    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 21:39:41.868754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:41.868727    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:39:41.869935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:41.869914    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:39:42.643501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:42.643468    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:39:42.644512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:39:42.644487    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:39:52.260052 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:39:52.260010    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:40:08.501831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:08.501639    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:40:08.588253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:08.588213    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:40:13.521871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.517314    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" podUID="c15bfba6-e818-4ca7-be86-c7e231166888" containerName="payload-processing" containerID="cri-o://6e86f5f13637d922e4c9cec9bd34185d1658163dcf0b97ec0acbbf13fb453ca7" gracePeriod=25
Nov 02 21:40:13.589368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.589291    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" podUID="68e0d317-90d6-4e68-8f4b-71185d7047da" containerName="payload-pre-processing" containerID="cri-o://fe595302edd8371694ba4e6e31e1dfce25c36dcf7b14bdede9f80240473189e7" gracePeriod=25
Nov 02 21:40:13.739257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.739220    4888 generic.go:356] "Generic (PLEG): container finished" podID="68e0d317-90d6-4e68-8f4b-71185d7047da" containerID="fe595302edd8371694ba4e6e31e1dfce25c36dcf7b14bdede9f80240473189e7" exitCode=0
Nov 02 21:40:13.739455 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.739394    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" event={"ID":"68e0d317-90d6-4e68-8f4b-71185d7047da","Type":"ContainerDied","Data":"fe595302edd8371694ba4e6e31e1dfce25c36dcf7b14bdede9f80240473189e7"}
Nov 02 21:40:13.741232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.741206    4888 generic.go:356] "Generic (PLEG): container finished" podID="c15bfba6-e818-4ca7-be86-c7e231166888" containerID="6e86f5f13637d922e4c9cec9bd34185d1658163dcf0b97ec0acbbf13fb453ca7" exitCode=0
Nov 02 21:40:13.741369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:13.741249    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" event={"ID":"c15bfba6-e818-4ca7-be86-c7e231166888","Type":"ContainerDied","Data":"6e86f5f13637d922e4c9cec9bd34185d1658163dcf0b97ec0acbbf13fb453ca7"}
Nov 02 21:40:14.087920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.086788    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:40:14.163424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.163392    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.252549    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\") pod \"68e0d317-90d6-4e68-8f4b-71185d7047da\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") "
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.252680    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\" (UniqueName: \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\") pod \"c15bfba6-e818-4ca7-be86-c7e231166888\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") "
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.252715    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\") pod \"c15bfba6-e818-4ca7-be86-c7e231166888\" (UID: \"c15bfba6-e818-4ca7-be86-c7e231166888\") "
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.252761    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\" (UniqueName: \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\") pod \"68e0d317-90d6-4e68-8f4b-71185d7047da\" (UID: \"68e0d317-90d6-4e68-8f4b-71185d7047da\") "
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.253144    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume" pod "68e0d317-90d6-4e68-8f4b-71185d7047da" (UID: "68e0d317-90d6-4e68-8f4b-71185d7047da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.253373    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e0d317-90d6-4e68-8f4b-71185d7047da-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.255320    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume" pod "c15bfba6-e818-4ca7-be86-c7e231166888" (UID: "c15bfba6-e818-4ca7-be86-c7e231166888"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.257203    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s" pod "68e0d317-90d6-4e68-8f4b-71185d7047da" (UID: "68e0d317-90d6-4e68-8f4b-71185d7047da"). InnerVolumeSpecName "kube-api-access-k654s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:14.259874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.259683    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29" pod "c15bfba6-e818-4ca7-be86-c7e231166888" (UID: "c15bfba6-e818-4ca7-be86-c7e231166888"). InnerVolumeSpecName "kube-api-access-p5b29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:14.355222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.355122    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5b29\" (UniqueName: \"kubernetes.io/projected/c15bfba6-e818-4ca7-be86-c7e231166888-kube-api-access-p5b29\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:14.355222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.355163    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15bfba6-e818-4ca7-be86-c7e231166888-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:14.355222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.355178    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k654s\" (UniqueName: \"kubernetes.io/projected/68e0d317-90d6-4e68-8f4b-71185d7047da-kube-api-access-k654s\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:14.749381 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.749346    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"
Nov 02 21:40:14.749892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.749371    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg" event={"ID":"68e0d317-90d6-4e68-8f4b-71185d7047da","Type":"ContainerDied","Data":"1d63197b9c32afd664571b72e0ca7a0b44dba5d7b728d69c25382a05dbc1b17c"}
Nov 02 21:40:14.749892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.749448    4888 scope.go:122] "RemoveContainer" containerID="fe595302edd8371694ba4e6e31e1dfce25c36dcf7b14bdede9f80240473189e7"
Nov 02 21:40:14.751207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.751179    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"
Nov 02 21:40:14.751321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.751215    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg" event={"ID":"c15bfba6-e818-4ca7-be86-c7e231166888","Type":"ContainerDied","Data":"2c0fd9e751ceeeab006a1d6069af79685673802a3c526c894936b9d887dd4e4b"}
Nov 02 21:40:14.760012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.759989    4888 scope.go:122] "RemoveContainer" containerID="6e86f5f13637d922e4c9cec9bd34185d1658163dcf0b97ec0acbbf13fb453ca7"
Nov 02 21:40:14.907201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.907165    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:40:14.959661 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:14.959614    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg"]
Nov 02 21:40:15.057943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:15.057907    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:40:15.061459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:15.061429    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg"]
Nov 02 21:40:16.601272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:16.601239    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68e0d317-90d6-4e68-8f4b-71185d7047da" path="/var/lib/kubelet/pods/68e0d317-90d6-4e68-8f4b-71185d7047da/volumes"
Nov 02 21:40:16.601711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:16.601695    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c15bfba6-e818-4ca7-be86-c7e231166888" path="/var/lib/kubelet/pods/c15bfba6-e818-4ca7-be86-c7e231166888/volumes"
Nov 02 21:40:17.422442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:17.422409    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:40:17.546882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:17.543910    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:40:21.583875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.574490    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:40:21.599432 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.599395    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2d61f536-8217-413b-a18b-9db80ca82c32" containerName="payload-pre-processing"
Nov 02 21:40:21.599702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.599680    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d61f536-8217-413b-a18b-9db80ca82c32" containerName="payload-pre-processing"
Nov 02 21:40:21.600526 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.600507    4888 cpu_manager.go:414] "containerMap: removing container" podUID="68e0d317-90d6-4e68-8f4b-71185d7047da" containerName="payload-pre-processing"
Nov 02 21:40:21.600660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.600646    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68e0d317-90d6-4e68-8f4b-71185d7047da" containerName="payload-pre-processing"
Nov 02 21:40:21.600742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.600733    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c15bfba6-e818-4ca7-be86-c7e231166888" containerName="payload-processing"
Nov 02 21:40:21.600821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.600811    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c15bfba6-e818-4ca7-be86-c7e231166888" containerName="payload-processing"
Nov 02 21:40:21.601158 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.601141    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="68e0d317-90d6-4e68-8f4b-71185d7047da" containerName="payload-pre-processing"
Nov 02 21:40:21.601265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.601254    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c15bfba6-e818-4ca7-be86-c7e231166888" containerName="payload-processing"
Nov 02 21:40:21.601424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.601412    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d61f536-8217-413b-a18b-9db80ca82c32" containerName="payload-pre-processing"
Nov 02 21:40:21.642895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.634233    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:40:21.642895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.635029    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.709602 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.705890    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-06366c-dockercfg-dvv84\""
Nov 02 21:40:21.756759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.755449    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.756759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.755553    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjzn\" (UniqueName: \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.758421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.758368    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-06366c\""
Nov 02 21:40:21.804705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.804666    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:40:21.857547 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.857461    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.859041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.859015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjzn\" (UniqueName: \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.860792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.858903    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:21.991094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:21.989156    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.011798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.011727    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjzn\" (UniqueName: \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\") pod \"payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:22.081895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.081457    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.081895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.081523    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kqrc\" (UniqueName: \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.082154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.081927    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:22.085536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.085511    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:40:22.186418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.186380    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.186679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.186658    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kqrc\" (UniqueName: \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.199904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.199874    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.295666 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.295625    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kqrc\" (UniqueName: \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\") pod \"payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.425889 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:40:22.425797    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:40:22.544965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.543335    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:22.544965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.543656    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" podUID="e40a058a-0816-4667-940b-63d4fc182573" containerName="payload-processing" containerID="cri-o://b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2" gracePeriod=25
Nov 02 21:40:22.544965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.544903    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" podUID="63837466-414b-43b3-9904-b1c96ebaf755" containerName="payload-pre-processing" containerID="cri-o://1e4781a3de2ce4a26c44d7d4aab4a8fd810a4cff86f5049f74d06e9481885a68" gracePeriod=25
Nov 02 21:40:22.882634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.882413    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:40:22.905313 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:22.905276    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7352c232_4c95_45fe_b964_bc9ed4a58210.slice/crio-60480c6fed7d08439c25ac684a61402d12aa86eaf0a6820397385cf1bb43d9a0 WatchSource:0}: Error finding container 60480c6fed7d08439c25ac684a61402d12aa86eaf0a6820397385cf1bb43d9a0: Status 404 returned error can't find the container with id 60480c6fed7d08439c25ac684a61402d12aa86eaf0a6820397385cf1bb43d9a0
Nov 02 21:40:22.990086 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:22.989629    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:40:23.164224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.121308    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:40:23.200417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.200333    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\") pod \"e40a058a-0816-4667-940b-63d4fc182573\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") "
Nov 02 21:40:23.200757 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.200736    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\" (UniqueName: \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\") pod \"e40a058a-0816-4667-940b-63d4fc182573\" (UID: \"e40a058a-0816-4667-940b-63d4fc182573\") "
Nov 02 21:40:23.202906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.202815    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume" pod "e40a058a-0816-4667-940b-63d4fc182573" (UID: "e40a058a-0816-4667-940b-63d4fc182573"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:23.204168 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:23.203926    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd98d189_ac09_4958_92bf_6e12289326d4.slice/crio-42a4e630623e22b56d8d72cc35bb98d55c591bcac2d39dba41ecee14e36ce20b WatchSource:0}: Error finding container 42a4e630623e22b56d8d72cc35bb98d55c591bcac2d39dba41ecee14e36ce20b: Status 404 returned error can't find the container with id 42a4e630623e22b56d8d72cc35bb98d55c591bcac2d39dba41ecee14e36ce20b
Nov 02 21:40:23.205676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.205649    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4" pod "e40a058a-0816-4667-940b-63d4fc182573" (UID: "e40a058a-0816-4667-940b-63d4fc182573"). InnerVolumeSpecName "kube-api-access-jjvw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:23.208092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.206253    4888 generic.go:356] "Generic (PLEG): container finished" podID="e40a058a-0816-4667-940b-63d4fc182573" containerID="b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2" exitCode=0
Nov 02 21:40:23.208092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.206345    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" event={"ID":"e40a058a-0816-4667-940b-63d4fc182573","Type":"ContainerDied","Data":"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"}
Nov 02 21:40:23.208092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.206396    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846" event={"ID":"e40a058a-0816-4667-940b-63d4fc182573","Type":"ContainerDied","Data":"35dc7c0d85bd71f2cd0871e6c9c36cb7c9890eade185e2194b3066af0d931e07"}
Nov 02 21:40:23.208092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.206426    4888 scope.go:122] "RemoveContainer" containerID="b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"
Nov 02 21:40:23.208092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.206577    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"
Nov 02 21:40:23.227407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.226841    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" event={"ID":"7352c232-4c95-45fe-b964-bc9ed4a58210","Type":"ContainerStarted","Data":"60480c6fed7d08439c25ac684a61402d12aa86eaf0a6820397385cf1bb43d9a0"}
Nov 02 21:40:23.230132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.229943    4888 generic.go:356] "Generic (PLEG): container finished" podID="63837466-414b-43b3-9904-b1c96ebaf755" containerID="1e4781a3de2ce4a26c44d7d4aab4a8fd810a4cff86f5049f74d06e9481885a68" exitCode=0
Nov 02 21:40:23.230132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.230041    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" event={"ID":"63837466-414b-43b3-9904-b1c96ebaf755","Type":"ContainerDied","Data":"1e4781a3de2ce4a26c44d7d4aab4a8fd810a4cff86f5049f74d06e9481885a68"}
Nov 02 21:40:23.344332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.343544    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40a058a-0816-4667-940b-63d4fc182573-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:23.344332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.343590    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjvw4\" (UniqueName: \"kubernetes.io/projected/e40a058a-0816-4667-940b-63d4fc182573-kube-api-access-jjvw4\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:23.414624 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.414589    4888 scope.go:122] "RemoveContainer" containerID="b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"
Nov 02 21:40:23.415999 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:40:23.415964    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2\": container with ID starting with b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2 not found: ID does not exist" containerID="b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"
Nov 02 21:40:23.418200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.417951    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2"} err="failed to get container status \"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2\": rpc error: code = NotFound desc = could not find container \"b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2\": container with ID starting with b6d53a762239a29e33a6d8600442df5d694fe84bf667f7dbb305844dceec34e2 not found: ID does not exist"
Nov 02 21:40:23.422599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.421788    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:40:23.425260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.425239    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:40:23.427196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.427159    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846"]
Nov 02 21:40:23.546311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.546262    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\") pod \"63837466-414b-43b3-9904-b1c96ebaf755\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") "
Nov 02 21:40:23.546667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.546382    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\" (UniqueName: \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\") pod \"63837466-414b-43b3-9904-b1c96ebaf755\" (UID: \"63837466-414b-43b3-9904-b1c96ebaf755\") "
Nov 02 21:40:23.547346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.547310    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume" pod "63837466-414b-43b3-9904-b1c96ebaf755" (UID: "63837466-414b-43b3-9904-b1c96ebaf755"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:23.547698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.547658    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63837466-414b-43b3-9904-b1c96ebaf755-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:23.549364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.549338    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8" pod "63837466-414b-43b3-9904-b1c96ebaf755" (UID: "63837466-414b-43b3-9904-b1c96ebaf755"). InnerVolumeSpecName "kube-api-access-k87q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:23.649513 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:23.649463    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k87q8\" (UniqueName: \"kubernetes.io/projected/63837466-414b-43b3-9904-b1c96ebaf755-kube-api-access-k87q8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:24.240176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.240134    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" event={"ID":"dd98d189-ac09-4958-92bf-6e12289326d4","Type":"ContainerStarted","Data":"0fbaa362490ac584b8f30f8957daa8bcf258da76e3e0d3e0a4001d6c2816bb35"}
Nov 02 21:40:24.240714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.240210    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:24.240714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.240238    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" event={"ID":"dd98d189-ac09-4958-92bf-6e12289326d4","Type":"ContainerStarted","Data":"42a4e630623e22b56d8d72cc35bb98d55c591bcac2d39dba41ecee14e36ce20b"}
Nov 02 21:40:24.243481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.243451    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" event={"ID":"7352c232-4c95-45fe-b964-bc9ed4a58210","Type":"ContainerStarted","Data":"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"}
Nov 02 21:40:24.243662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.243532    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:24.244971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.244944    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"
Nov 02 21:40:24.245108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.244968    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc" event={"ID":"63837466-414b-43b3-9904-b1c96ebaf755","Type":"ContainerDied","Data":"95ae146dfde95a2df2b6bd81f56ffb417f58c7d064c475181b2549186f10e7f7"}
Nov 02 21:40:24.245108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.245025    4888 scope.go:122] "RemoveContainer" containerID="1e4781a3de2ce4a26c44d7d4aab4a8fd810a4cff86f5049f74d06e9481885a68"
Nov 02 21:40:24.352424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.352316    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" podStartSLOduration=3.35229796 podStartE2EDuration="3.35229796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:24.348897551 +0000 UTC m=+1653.892689469" watchObservedRunningTime="2026-11-02 21:40:24.35229796 +0000 UTC m=+1653.896089880"
Nov 02 21:40:24.459282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.459192    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" podStartSLOduration=3.459172143 podStartE2EDuration="3.459172143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:24.458754865 +0000 UTC m=+1654.002546781" watchObservedRunningTime="2026-11-02 21:40:24.459172143 +0000 UTC m=+1654.002964060"
Nov 02 21:40:24.575872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.575549    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:40:24.585511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.581931    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc"]
Nov 02 21:40:24.644764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.644679    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63837466-414b-43b3-9904-b1c96ebaf755" path="/var/lib/kubelet/pods/63837466-414b-43b3-9904-b1c96ebaf755/volumes"
Nov 02 21:40:24.645207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.645192    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e40a058a-0816-4667-940b-63d4fc182573" path="/var/lib/kubelet/pods/e40a058a-0816-4667-940b-63d4fc182573/volumes"
Nov 02 21:40:24.764055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764018    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:40:24.764784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764758    4888 cpu_manager.go:414] "containerMap: removing container" podUID="63837466-414b-43b3-9904-b1c96ebaf755" containerName="payload-pre-processing"
Nov 02 21:40:24.764784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764778    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63837466-414b-43b3-9904-b1c96ebaf755" containerName="payload-pre-processing"
Nov 02 21:40:24.765014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764804    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e40a058a-0816-4667-940b-63d4fc182573" containerName="payload-processing"
Nov 02 21:40:24.765014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764810    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e40a058a-0816-4667-940b-63d4fc182573" containerName="payload-processing"
Nov 02 21:40:24.765014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764926    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="63837466-414b-43b3-9904-b1c96ebaf755" containerName="payload-pre-processing"
Nov 02 21:40:24.765014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.764939    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e40a058a-0816-4667-940b-63d4fc182573" containerName="payload-processing"
Nov 02 21:40:24.769955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.769931    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.821064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.820308    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d3b375-openshift-default-dockercfg-lbnjk\""
Nov 02 21:40:24.824500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.824465    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:40:24.932645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.932597    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.932877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.932731    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.932877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.932782    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwzn9\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.932877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.932857    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.933094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.933071    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.933329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.933183    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.933329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.933263    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.933470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.933344    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:24.933470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:24.933377    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.037129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.037096    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.039156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.038283    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.042500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.042471    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.054985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.042885    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.056421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.055705    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.057128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.057102    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.057313 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.056664    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.057480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.057454    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwzn9\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.069368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.068711    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.069654 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.069634    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.070277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.070249    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.070778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.070754    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.083155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.069190    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.083378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.070029    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.083501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.075262    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.095353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.095316    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.198934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.195112    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwzn9\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.226884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.224581    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\") pod \"e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") " pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:25.402011 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:25.391483    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:26.277861 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.277803    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:40:26.278459 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:26.278432    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode977df1a_c89d_45e5_b0e1_499fa3ad2f54.slice/crio-45262bb68410adb96d941510c5fc5c4ce5d99a6353dea7ad07bc1a62b5833bff WatchSource:0}: Error finding container 45262bb68410adb96d941510c5fc5c4ce5d99a6353dea7ad07bc1a62b5833bff: Status 404 returned error can't find the container with id 45262bb68410adb96d941510c5fc5c4ce5d99a6353dea7ad07bc1a62b5833bff
Nov 02 21:40:26.282407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.282362    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:26.282523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.282454    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:26.282523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.282515    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:26.487477 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.487435    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerStarted","Data":"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"}
Nov 02 21:40:26.488041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.487538    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerStarted","Data":"45262bb68410adb96d941510c5fc5c4ce5d99a6353dea7ad07bc1a62b5833bff"}
Nov 02 21:40:26.601032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:26.600911    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podStartSLOduration=2.60082889 podStartE2EDuration="2.60082889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:26.598717489 +0000 UTC m=+1656.142509410" watchObservedRunningTime="2026-11-02 21:40:26.60082889 +0000 UTC m=+1656.144620809"
Nov 02 21:40:27.391986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:27.391938    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:27.398891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:27.398833    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:27.494298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:27.494252    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:27.495361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:27.495336    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:40:31.323495 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.323282    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:40:31.520870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.516131    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.682880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.656933    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b7107c\""
Nov 02 21:40:31.700257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.656944    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b7107c-dockercfg-vdz8z\""
Nov 02 21:40:31.700479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.657090    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:40:31.751928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.718492    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlm7q\" (UniqueName: \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.790150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.790098    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.791322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.751900    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:40:31.893284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.893059    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.894693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.894656    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlm7q\" (UniqueName: \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.897712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.897668    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:40:31.898639 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.898617    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:31.941661 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.939835    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:31.971135 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:31.971045    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlm7q\" (UniqueName: \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\") pod \"payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:32.005052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.005016    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.005732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.005708    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc69z\" (UniqueName: \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.094895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.094836    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:32.109323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.109282    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc69z\" (UniqueName: \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.109485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.109340    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.112270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.112240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.168300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.168249    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc69z\" (UniqueName: \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\") pod \"payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") " pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.315004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.314835    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:32.479090 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:32.479035    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf948c30_2b3a_4bd3_96ca_7d240901b419.slice/crio-cca4653882f6cc47613cbf843a60c503a1d9ff518bc355775792bb4eb8300fd8 WatchSource:0}: Error finding container cca4653882f6cc47613cbf843a60c503a1d9ff518bc355775792bb4eb8300fd8: Status 404 returned error can't find the container with id cca4653882f6cc47613cbf843a60c503a1d9ff518bc355775792bb4eb8300fd8
Nov 02 21:40:32.480662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.480633    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:40:32.779921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.779873    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" event={"ID":"af948c30-2b3a-4bd3-96ca-7d240901b419","Type":"ContainerStarted","Data":"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"}
Nov 02 21:40:32.780043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.779956    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" event={"ID":"af948c30-2b3a-4bd3-96ca-7d240901b419","Type":"ContainerStarted","Data":"cca4653882f6cc47613cbf843a60c503a1d9ff518bc355775792bb4eb8300fd8"}
Nov 02 21:40:32.780043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.780001    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:32.781761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.781727    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:40:32.888385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:32.888321    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" podStartSLOduration=1.888300447 podStartE2EDuration="1.888300447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:32.887410218 +0000 UTC m=+1662.431202149" watchObservedRunningTime="2026-11-02 21:40:32.888300447 +0000 UTC m=+1662.432092364"
Nov 02 21:40:33.816887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:33.812530    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" event={"ID":"599086a4-3738-4f77-9dae-f442b7062008","Type":"ContainerStarted","Data":"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"}
Nov 02 21:40:33.816887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:33.812592    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" event={"ID":"599086a4-3738-4f77-9dae-f442b7062008","Type":"ContainerStarted","Data":"63c977a627ed00a6661f906c95fade071ddf25566a2ceec16b1fea5e68ad0550"}
Nov 02 21:40:33.816887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:33.813199    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:33.931073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:33.931002    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" podStartSLOduration=2.9309799290000003 podStartE2EDuration="2.930979929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:33.930143311 +0000 UTC m=+1663.473935228" watchObservedRunningTime="2026-11-02 21:40:33.930979929 +0000 UTC m=+1663.474771846"
Nov 02 21:40:35.479463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:35.479434    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:40:35.481103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:35.481083    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:40:40.296007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.295944    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:40:40.325875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.316317    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.380439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.379541    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:40:40.466878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.439139    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvvxf\" (UniqueName: \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.539877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.533031    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.585929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.484042    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d3b375-dockercfg-q8d6b\""
Nov 02 21:40:40.586178 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.494503    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:40:40.606084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.483980    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d3b375\""
Nov 02 21:40:40.679872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.669157    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvvxf\" (UniqueName: \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.680245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.680206    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.716967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.715197    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.751873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.735151    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:40:40.766683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.752214    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:40.921897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.912462    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvvxf\" (UniqueName: \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\") pod \"payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:40.921897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.914568    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jptb\" (UniqueName: \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:40.921897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:40.914843    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.016276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.016228    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jptb\" (UniqueName: \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.016499 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.016370    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.017161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.017139    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.070773 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.070735    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jptb\" (UniqueName: \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\") pod \"payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") " pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.177924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.174422    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:41.243466 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.243158    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.673907 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:41.673875    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2e2563_37ef_44af_b64d_6a50d34966cf.slice/crio-9845abb10bb8f9be8b669dc715eeba4f553cc8569eff34c201331d2f8df74ee1 WatchSource:0}: Error finding container 9845abb10bb8f9be8b669dc715eeba4f553cc8569eff34c201331d2f8df74ee1: Status 404 returned error can't find the container with id 9845abb10bb8f9be8b669dc715eeba4f553cc8569eff34c201331d2f8df74ee1
Nov 02 21:40:41.674535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.674504    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:40:41.725614 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:41.725583    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c2edc5_4fc0_4271_8de8_16c09325d313.slice/crio-02a84e982a7c4639e1c52930b08c21e8cd79112197b09b9268b5941daa1ff1e9 WatchSource:0}: Error finding container 02a84e982a7c4639e1c52930b08c21e8cd79112197b09b9268b5941daa1ff1e9: Status 404 returned error can't find the container with id 02a84e982a7c4639e1c52930b08c21e8cd79112197b09b9268b5941daa1ff1e9
Nov 02 21:40:41.726078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.726046    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:40:41.922325 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.922279    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" event={"ID":"82c2edc5-4fc0-4271-8de8-16c09325d313","Type":"ContainerStarted","Data":"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"}
Nov 02 21:40:41.922512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.922334    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" event={"ID":"82c2edc5-4fc0-4271-8de8-16c09325d313","Type":"ContainerStarted","Data":"02a84e982a7c4639e1c52930b08c21e8cd79112197b09b9268b5941daa1ff1e9"}
Nov 02 21:40:41.922512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.922385    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:41.925610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.925583    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" event={"ID":"5e2e2563-37ef-44af-b64d-6a50d34966cf","Type":"ContainerStarted","Data":"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"}
Nov 02 21:40:41.925740 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.925636    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:41.925740 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:41.925652    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" event={"ID":"5e2e2563-37ef-44af-b64d-6a50d34966cf","Type":"ContainerStarted","Data":"9845abb10bb8f9be8b669dc715eeba4f553cc8569eff34c201331d2f8df74ee1"}
Nov 02 21:40:42.031490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:42.031379    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" podStartSLOduration=2.031365564 podStartE2EDuration="2.031365564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:42.030109294 +0000 UTC m=+1671.573901214" watchObservedRunningTime="2026-11-02 21:40:42.031365564 +0000 UTC m=+1671.575157479"
Nov 02 21:40:42.141424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:42.141356    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" podStartSLOduration=2.141340524 podStartE2EDuration="2.141340524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:42.138144195 +0000 UTC m=+1671.681936111" watchObservedRunningTime="2026-11-02 21:40:42.141340524 +0000 UTC m=+1671.685132439"
Nov 02 21:40:43.815974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:43.815945    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:40:44.829514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:44.829487    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:40:49.846494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.846455    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:40:49.847007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.846706    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" podUID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" containerName="maas-api" containerID="cri-o://0093dba5bdcae5cde6d18d23be0902c3b84f2ecc368c7a8f695423e036c75474" gracePeriod=30
Nov 02 21:40:49.956010 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.955969    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:40:49.956303 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.956249    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy" containerID="cri-o://10356d08420322b3795eae4dda2053b5d9402fc6c68a6e70812f93092169ba19" gracePeriod=30
Nov 02 21:40:49.961084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.961048    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:40:49.961315 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.961286    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy" containerID="cri-o://4daee9dd5131de3584b5b700710124699e4c52fe3617f88eb2c64da166f74113" gracePeriod=30
Nov 02 21:40:49.967521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.967476    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:40:49.967972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.967923    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy" containerID="cri-o://369f0f12f21374ea0ff2a90fdcae1243f1342e9483d786c64048eab2bc9cb909" gracePeriod=30
Nov 02 21:40:49.987695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.987654    4888 generic.go:356] "Generic (PLEG): container finished" podID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" containerID="0093dba5bdcae5cde6d18d23be0902c3b84f2ecc368c7a8f695423e036c75474" exitCode=0
Nov 02 21:40:49.987964 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:49.987745    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" event={"ID":"bb147d9c-8b20-45df-890f-ccc3468ac5a3","Type":"ContainerDied","Data":"0093dba5bdcae5cde6d18d23be0902c3b84f2ecc368c7a8f695423e036c75474"}
Nov 02 21:40:50.022816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.022311    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:40:50.022816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.022590    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" containerID="cri-o://cfb2f7f99d9fc9e05fad9873045aaec440a58026664b2dbcc6f80d1552a71595" gracePeriod=30
Nov 02 21:40:50.493108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.493070    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:40:50.577648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.577591    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\") pod \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") "
Nov 02 21:40:50.577900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.577661    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\" (UniqueName: \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\") pod \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") "
Nov 02 21:40:50.577900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.577769    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\") pod \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") "
Nov 02 21:40:50.577900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.577827    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\") pod \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\" (UID: \"bb147d9c-8b20-45df-890f-ccc3468ac5a3\") "
Nov 02 21:40:50.580944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.579498    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca" pod "bb147d9c-8b20-45df-890f-ccc3468ac5a3" (UID: "bb147d9c-8b20-45df-890f-ccc3468ac5a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:50.728647 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.696999    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb147d9c-8b20-45df-890f-ccc3468ac5a3-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:50.747216 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.747176    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls" pod "bb147d9c-8b20-45df-890f-ccc3468ac5a3" (UID: "bb147d9c-8b20-45df-890f-ccc3468ac5a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:40:50.758448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.758384    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs" pod "bb147d9c-8b20-45df-890f-ccc3468ac5a3" (UID: "bb147d9c-8b20-45df-890f-ccc3468ac5a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:40:50.763705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.760519    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4" pod "bb147d9c-8b20-45df-890f-ccc3468ac5a3" (UID: "bb147d9c-8b20-45df-890f-ccc3468ac5a3"). InnerVolumeSpecName "kube-api-access-b8tr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:50.799030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.798995    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:50.799030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.799026    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8tr4\" (UniqueName: \"kubernetes.io/projected/bb147d9c-8b20-45df-890f-ccc3468ac5a3-kube-api-access-b8tr4\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:50.799030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:50.799037    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb147d9c-8b20-45df-890f-ccc3468ac5a3-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:51.084903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.083931    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr" event={"ID":"bb147d9c-8b20-45df-890f-ccc3468ac5a3","Type":"ContainerDied","Data":"5774cf4ac8506ec3aca96f0322981bb604e626e86ead0c14203b6be6f3ce774c"}
Nov 02 21:40:51.084903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.084011    4888 scope.go:122] "RemoveContainer" containerID="0093dba5bdcae5cde6d18d23be0902c3b84f2ecc368c7a8f695423e036c75474"
Nov 02 21:40:51.084903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.084180    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"
Nov 02 21:40:51.382067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.381978    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:40:51.394978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.389424    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5935b530-68d98cc4fc-8q4zr"]
Nov 02 21:40:51.984038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.984003    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:40:51.984276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:51.984254    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy" containerID="cri-o://dd43d8d26818e4014b8e60aa47a6616d4ba8ed36d8658b0adc269bad2fc3615d" gracePeriod=30
Nov 02 21:40:52.473914 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:40:52.468473    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:40:52.602428 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:52.602396    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" path="/var/lib/kubelet/pods/bb147d9c-8b20-45df-890f-ccc3468ac5a3/volumes"
Nov 02 21:40:52.750027 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:52.749931    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:40:52.750209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:52.750162    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" containerID="cri-o://0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215" gracePeriod=30
Nov 02 21:40:52.985874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:52.983055    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:40:53.014929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:53.012058    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:40:54.272133 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:54.272100    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:40:54.272763 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:54.272731    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:40:55.277992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.277759    4888 generic.go:356] "Generic (PLEG): container finished" podID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerID="10356d08420322b3795eae4dda2053b5d9402fc6c68a6e70812f93092169ba19" exitCode=0
Nov 02 21:40:55.277992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.277909    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" event={"ID":"d072b7ab-499d-4b3a-8826-8d35d52ffaf0","Type":"ContainerDied","Data":"10356d08420322b3795eae4dda2053b5d9402fc6c68a6e70812f93092169ba19"}
Nov 02 21:40:55.282276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.281606    4888 generic.go:356] "Generic (PLEG): container finished" podID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerID="cfb2f7f99d9fc9e05fad9873045aaec440a58026664b2dbcc6f80d1552a71595" exitCode=0
Nov 02 21:40:55.282276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.281706    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" event={"ID":"024f1b76-1c00-4b3f-94da-c38165e6c8f8","Type":"ContainerDied","Data":"cfb2f7f99d9fc9e05fad9873045aaec440a58026664b2dbcc6f80d1552a71595"}
Nov 02 21:40:55.284937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.284907    4888 generic.go:356] "Generic (PLEG): container finished" podID="547be484-ab43-4f26-9f3a-f44c5674727b" containerID="4daee9dd5131de3584b5b700710124699e4c52fe3617f88eb2c64da166f74113" exitCode=0
Nov 02 21:40:55.285102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.285081    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" event={"ID":"547be484-ab43-4f26-9f3a-f44c5674727b","Type":"ContainerDied","Data":"4daee9dd5131de3584b5b700710124699e4c52fe3617f88eb2c64da166f74113"}
Nov 02 21:40:55.286893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.286827    4888 generic.go:356] "Generic (PLEG): container finished" podID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerID="369f0f12f21374ea0ff2a90fdcae1243f1342e9483d786c64048eab2bc9cb909" exitCode=0
Nov 02 21:40:55.287014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.286931    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" event={"ID":"e54a745f-4dd9-4b1d-ad14-580491ac1637","Type":"ContainerDied","Data":"369f0f12f21374ea0ff2a90fdcae1243f1342e9483d786c64048eab2bc9cb909"}
Nov 02 21:40:55.361115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.359462    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:40:55.393755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.393723    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:40:55.414332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.414303    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:40:55.484815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.484658    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.484815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.484732    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.485369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485162    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.485369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485219    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.485369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485254    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.485369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485327    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.485783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485757    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.485945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485819    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.485945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485888    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485946    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.485975    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486007    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486043    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486084    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.486287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486121    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.486287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486166    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486202    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.486489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486295    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486363    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486399    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486435    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486489    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486537    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.486698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486614    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\") pod \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\" (UID: \"d072b7ab-499d-4b3a-8826-8d35d52ffaf0\") "
Nov 02 21:40:55.486698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486658    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.486945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486708    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\") pod \"547be484-ab43-4f26-9f3a-f44c5674727b\" (UID: \"547be484-ab43-4f26-9f3a-f44c5674727b\") "
Nov 02 21:40:55.486945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.486751    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\") pod \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\" (UID: \"024f1b76-1c00-4b3f-94da-c38165e6c8f8\") "
Nov 02 21:40:55.488503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.488471    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.489154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.488927    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.489376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.489348    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.490021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.489839    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:55.490120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.490013    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.490459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.490401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.490737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.490717    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:55.492163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492137    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492315    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492416    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492500    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492535    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492740    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.492900    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:55.497566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.493435    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.515926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.515822    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:40:55.516164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.516132    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" containerID="cri-o://c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342" gracePeriod=30
Nov 02 21:40:55.517194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.517159    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "kube-api-access-jrzm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.517757 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.517717    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "kube-api-access-c5s87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.517928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.517874    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.518429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.518405    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.524409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.524349    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "kube-api-access-wlkw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.531555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.531482    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.531933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.531887    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.532030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.531934    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:55.532030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.531936    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:55.532307 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.532267    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo" pod "024f1b76-1c00-4b3f-94da-c38165e6c8f8" (UID: "024f1b76-1c00-4b3f-94da-c38165e6c8f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:55.533662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.533622    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo" pod "d072b7ab-499d-4b3a-8826-8d35d52ffaf0" (UID: "d072b7ab-499d-4b3a-8826-8d35d52ffaf0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:55.535292 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.535261    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo" pod "547be484-ab43-4f26-9f3a-f44c5674727b" (UID: "547be484-ab43-4f26-9f3a-f44c5674727b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:55.589676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589639    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589677    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589692    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrzm8\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-kube-api-access-jrzm8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589707    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589720    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589734    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589748    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589760    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589773    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589786    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547be484-ab43-4f26-9f3a-f44c5674727b-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589801    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589815    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589828    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547be484-ab43-4f26-9f3a-f44c5674727b-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589842    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589873    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589887    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589900    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589913    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547be484-ab43-4f26-9f3a-f44c5674727b-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.589917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589927    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589943    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589956    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589967    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589982    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlkw4\" (UniqueName: \"kubernetes.io/projected/024f1b76-1c00-4b3f-94da-c38165e6c8f8-kube-api-access-wlkw4\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.589995    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5s87\" (UniqueName: \"kubernetes.io/projected/d072b7ab-499d-4b3a-8826-8d35d52ffaf0-kube-api-access-c5s87\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.590009    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.590022    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547be484-ab43-4f26-9f3a-f44c5674727b-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:55.590430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:55.590035    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f1b76-1c00-4b3f-94da-c38165e6c8f8-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.105806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.105772    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:40:56.229377 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229333    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.229580 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229435    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.229580 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229495    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.229706 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229616    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.229766 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229710    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.229818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.229781    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.230176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230146    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.230330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230243    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.230407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230355    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\") pod \"e54a745f-4dd9-4b1d-ad14-580491ac1637\" (UID: \"e54a745f-4dd9-4b1d-ad14-580491ac1637\") "
Nov 02 21:40:56.230565 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230533    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:56.230660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230572    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:56.230811 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230789    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:56.231014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.230993    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:56.231209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.231190    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.231312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.231215    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e54a745f-4dd9-4b1d-ad14-580491ac1637-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.231312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.231231    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.231312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.231244    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.231312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.231284    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:56.233640 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.233615    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:56.236025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.235987    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "kube-api-access-6b6w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:56.237015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.236989    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:56.237842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.237811    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token" pod "e54a745f-4dd9-4b1d-ad14-580491ac1637" (UID: "e54a745f-4dd9-4b1d-ad14-580491ac1637"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:56.297183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.297089    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"
Nov 02 21:40:56.298031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.297994    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb" event={"ID":"e54a745f-4dd9-4b1d-ad14-580491ac1637","Type":"ContainerDied","Data":"7894cc2193395fe0455b44306ffb6ade3751dfdbd18ad5a61c8461e893fbbf37"}
Nov 02 21:40:56.298272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.298251    4888 scope.go:122] "RemoveContainer" containerID="369f0f12f21374ea0ff2a90fdcae1243f1342e9483d786c64048eab2bc9cb909"
Nov 02 21:40:56.299019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.298974    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"
Nov 02 21:40:56.299019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.298987    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f" event={"ID":"d072b7ab-499d-4b3a-8826-8d35d52ffaf0","Type":"ContainerDied","Data":"04bcd798c588487e4ad8cd3d520bc46951dbb55189c5c519e7c434e1b42706d8"}
Nov 02 21:40:56.301051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.301019    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf" event={"ID":"024f1b76-1c00-4b3f-94da-c38165e6c8f8","Type":"ContainerDied","Data":"eee8bcdfc3513cff94c3adb29bf012f5afbec106b32b47baa59da0b6a3bfb136"}
Nov 02 21:40:56.301143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.301049    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"
Nov 02 21:40:56.302963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.302939    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"
Nov 02 21:40:56.303070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.303048    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x" event={"ID":"547be484-ab43-4f26-9f3a-f44c5674727b","Type":"ContainerDied","Data":"65812e59faba5c57a5ea4a7dbc6201d0cc13ce40aec55ac97de3f94ac4bf123c"}
Nov 02 21:40:56.317950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.317929    4888 scope.go:122] "RemoveContainer" containerID="10356d08420322b3795eae4dda2053b5d9402fc6c68a6e70812f93092169ba19"
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.338693    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.338731    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.338748    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.338764    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b6w8\" (UniqueName: \"kubernetes.io/projected/e54a745f-4dd9-4b1d-ad14-580491ac1637-kube-api-access-6b6w8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.338780    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e54a745f-4dd9-4b1d-ad14-580491ac1637-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:56.401884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.387116    4888 scope.go:122] "RemoveContainer" containerID="cfb2f7f99d9fc9e05fad9873045aaec440a58026664b2dbcc6f80d1552a71595"
Nov 02 21:40:56.453528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.453501    4888 scope.go:122] "RemoveContainer" containerID="4daee9dd5131de3584b5b700710124699e4c52fe3617f88eb2c64da166f74113"
Nov 02 21:40:56.508456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.508424    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:40:56.514562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.514530    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f"]
Nov 02 21:40:56.601980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.601901    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" path="/var/lib/kubelet/pods/d072b7ab-499d-4b3a-8826-8d35d52ffaf0/volumes"
Nov 02 21:40:56.613917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.613877    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:40:56.620253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.620219    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf"]
Nov 02 21:40:56.723077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.723042    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:40:56.726894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.726826    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x"]
Nov 02 21:40:56.824638 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.824602    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:40:56.825367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825345    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy"
Nov 02 21:40:56.825367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825364    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825388    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" containerName="maas-api"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825396    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" containerName="maas-api"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825429    4888 cpu_manager.go:414] "containerMap: removing container" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825435    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825448    4888 cpu_manager.go:414] "containerMap: removing container" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825453    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825461    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy"
Nov 02 21:40:56.825567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825468    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy"
Nov 02 21:40:56.825993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825573    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" containerName="istio-proxy"
Nov 02 21:40:56.825993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825584    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d072b7ab-499d-4b3a-8826-8d35d52ffaf0" containerName="istio-proxy"
Nov 02 21:40:56.825993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825592    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" containerName="istio-proxy"
Nov 02 21:40:56.825993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825601    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb147d9c-8b20-45df-890f-ccc3468ac5a3" containerName="maas-api"
Nov 02 21:40:56.825993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.825611    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" containerName="istio-proxy"
Nov 02 21:40:56.832824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.832796    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.835553 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.835523    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:40:56.839592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.839563    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb"]
Nov 02 21:40:56.887119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.887037    4888 patch_prober.go:29] interesting pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:40:56.887119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.887102    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:40:56.889084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.889044    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:40:56.954946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.954891    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.954946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.954953    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955072    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955118    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955331 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955210    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw5tv\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955422    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955465    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.955618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.955597    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:56.970566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:56.970522    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-d4d045-openshift-default-dockercfg-pqjb2\""
Nov 02 21:40:57.058184 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058076    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058184 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058173    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058222    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058255    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw5tv\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058390    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058444 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058420    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058461    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058574    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.058771 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.058691    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.060436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.060408    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.060819 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.060796    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.061807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.061605    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.062204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.062181    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.063169 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.063142    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.065588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.065567    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.087826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.087790    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.133766 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.133626    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.147107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.146988    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw5tv\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\") pod \"e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") " pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.161665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.161321    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:57.404033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.403765    4888 generic.go:356] "Generic (PLEG): container finished" podID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerID="dd43d8d26818e4014b8e60aa47a6616d4ba8ed36d8658b0adc269bad2fc3615d" exitCode=0
Nov 02 21:40:57.404033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.403880    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" event={"ID":"8ac93a91-7851-42ea-b49e-6d4db406ee0c","Type":"ContainerDied","Data":"dd43d8d26818e4014b8e60aa47a6616d4ba8ed36d8658b0adc269bad2fc3615d"}
Nov 02 21:40:57.637436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.637040    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:40:57.646432 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.646404    4888 patch_prober.go:29] interesting pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:40:57.646545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.646457    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:40:57.684453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684349    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684417    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684460    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684544    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684597    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684611    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:57.684935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684725    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:57.684935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684877    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.684935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684913    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.685091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.684965    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.685091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685038    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\") pod \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\" (UID: \"8ac93a91-7851-42ea-b49e-6d4db406ee0c\") "
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685410    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685445    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685600    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685779    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685791    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685802    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685810    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.685865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.685819    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.687772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.687730    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:57.688686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.688662    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:57.689385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.689361    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:57.690192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.690172    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62" pod "8ac93a91-7851-42ea-b49e-6d4db406ee0c" (UID: "8ac93a91-7851-42ea-b49e-6d4db406ee0c"). InnerVolumeSpecName "kube-api-access-xtb62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:57.787821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.787784    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.787821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.787818    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.788091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.787831    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ac93a91-7851-42ea-b49e-6d4db406ee0c-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.788091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.787867    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtb62\" (UniqueName: \"kubernetes.io/projected/8ac93a91-7851-42ea-b49e-6d4db406ee0c-kube-api-access-xtb62\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:57.977186 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:40:57.976801    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f46504_204b_47b6_9fdc_e3f41b89cdbb.slice/crio-eec84a293de33f78966a38bac3660d0ae2fd2c125b0a17e8cb0c36bf7441572c WatchSource:0}: Error finding container eec84a293de33f78966a38bac3660d0ae2fd2c125b0a17e8cb0c36bf7441572c: Status 404 returned error can't find the container with id eec84a293de33f78966a38bac3660d0ae2fd2c125b0a17e8cb0c36bf7441572c
Nov 02 21:40:57.980672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.980120    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:57.980672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.980217    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:57.980672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.980259    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:40:57.982665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:57.982632    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:40:58.097775 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.097736    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:40:58.193891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.193831    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.194627 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.194598    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.194755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.194696    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.194926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.194897    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:58.195041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.195018    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:58.195159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.195142    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.195617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.195593    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:40:58.195800 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.195783    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.196113 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196092    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:58.196223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196206    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.196525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196507    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.196726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196509    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:58.196799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196719    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:40:58.196975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.196960    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.197435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.197408    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\") pod \"48fc9342-0659-42f5-b0ed-5475149426da\" (UID: \"48fc9342-0659-42f5-b0ed-5475149426da\") "
Nov 02 21:40:58.198503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.198484    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.198632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.198615    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.198739 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.198727    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48fc9342-0659-42f5-b0ed-5475149426da-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.198821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.198810    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.199115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.199085    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:58.199196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.199185    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.199270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.199259    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48fc9342-0659-42f5-b0ed-5475149426da-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.199688 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.199660    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "kube-api-access-tr5rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:40:58.201530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.201497    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo" pod "48fc9342-0659-42f5-b0ed-5475149426da" (UID: "48fc9342-0659-42f5-b0ed-5475149426da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:40:58.301571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.301523    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.301571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.301569    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr5rt\" (UniqueName: \"kubernetes.io/projected/48fc9342-0659-42f5-b0ed-5475149426da-kube-api-access-tr5rt\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.301841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.301585    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48fc9342-0659-42f5-b0ed-5475149426da-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:40:58.445593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.445554    4888 generic.go:356] "Generic (PLEG): container finished" podID="48fc9342-0659-42f5-b0ed-5475149426da" containerID="0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215" exitCode=0
Nov 02 21:40:58.446109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.445612    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"
Nov 02 21:40:58.446109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.445640    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerDied","Data":"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"}
Nov 02 21:40:58.446109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.445705    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp" event={"ID":"48fc9342-0659-42f5-b0ed-5475149426da","Type":"ContainerDied","Data":"e395366c7701980a9177010fccaadbd83a5a7849f7c50ae221d9e6dba3d2283e"}
Nov 02 21:40:58.446109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.445739    4888 scope.go:122] "RemoveContainer" containerID="0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"
Nov 02 21:40:58.447899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.447870    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9" event={"ID":"8ac93a91-7851-42ea-b49e-6d4db406ee0c","Type":"ContainerDied","Data":"6193ed9eb1b348b6e4e88c5b40038ea05fb0ac60b99feefad5c9ef93027901b3"}
Nov 02 21:40:58.448111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.447936    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"
Nov 02 21:40:58.449977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.449892    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerStarted","Data":"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"}
Nov 02 21:40:58.449977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.449945    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerStarted","Data":"eec84a293de33f78966a38bac3660d0ae2fd2c125b0a17e8cb0c36bf7441572c"}
Nov 02 21:40:58.461133 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.460971    4888 scope.go:122] "RemoveContainer" containerID="3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"
Nov 02 21:40:58.473961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.473933    4888 scope.go:122] "RemoveContainer" containerID="0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"
Nov 02 21:40:58.474332 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:40:58.474304    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215\": container with ID starting with 0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215 not found: ID does not exist" containerID="0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"
Nov 02 21:40:58.474416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.474343    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215"} err="failed to get container status \"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215\": rpc error: code = NotFound desc = could not find container \"0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215\": container with ID starting with 0b25a181242d66d8120682d4954c9a52560c2fabbb08d8ce704f6ed9170f6215 not found: ID does not exist"
Nov 02 21:40:58.474416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.474362    4888 scope.go:122] "RemoveContainer" containerID="3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"
Nov 02 21:40:58.474640 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:40:58.474625    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0\": container with ID starting with 3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0 not found: ID does not exist" containerID="3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"
Nov 02 21:40:58.474687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.474645    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0"} err="failed to get container status \"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0\": rpc error: code = NotFound desc = could not find container \"3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0\": container with ID starting with 3dfd277b8d0e412c780ae949f167e101802f9695b6d892ad212a6425fbe350e0 not found: ID does not exist"
Nov 02 21:40:58.474687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.474659    4888 scope.go:122] "RemoveContainer" containerID="dd43d8d26818e4014b8e60aa47a6616d4ba8ed36d8658b0adc269bad2fc3615d"
Nov 02 21:40:58.602958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.602925    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="024f1b76-1c00-4b3f-94da-c38165e6c8f8" path="/var/lib/kubelet/pods/024f1b76-1c00-4b3f-94da-c38165e6c8f8/volumes"
Nov 02 21:40:58.603863 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.603823    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="547be484-ab43-4f26-9f3a-f44c5674727b" path="/var/lib/kubelet/pods/547be484-ab43-4f26-9f3a-f44c5674727b/volumes"
Nov 02 21:40:58.604490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.604456    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e54a745f-4dd9-4b1d-ad14-580491ac1637" path="/var/lib/kubelet/pods/e54a745f-4dd9-4b1d-ad14-580491ac1637/volumes"
Nov 02 21:40:58.605214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.605196    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:40:58.650946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.650910    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp"]
Nov 02 21:40:58.957207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.957134    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podStartSLOduration=2.9571124810000002 podStartE2EDuration="2.957112481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:58.766720625 +0000 UTC m=+1688.310512542" watchObservedRunningTime="2026-11-02 21:40:58.957112481 +0000 UTC m=+1688.500904414"
Nov 02 21:40:58.991882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:58.972902    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:40:59.024361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:59.020707    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9"]
Nov 02 21:40:59.162105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:59.162065    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:59.168460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:59.168434    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:59.459612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:59.459582    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:40:59.460723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:40:59.460705    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:41:00.603950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.603914    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48fc9342-0659-42f5-b0ed-5475149426da" path="/var/lib/kubelet/pods/48fc9342-0659-42f5-b0ed-5475149426da/volumes"
Nov 02 21:41:00.604703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.604683    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" path="/var/lib/kubelet/pods/8ac93a91-7851-42ea-b49e-6d4db406ee0c/volumes"
Nov 02 21:41:00.809244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.809180    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:41:00.881990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.881955    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882055    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882113    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882192    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882240    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882335    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882371    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882388    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:00.882549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882417    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882445    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\") pod \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\" (UID: \"c148f0bf-fd9c-4803-8c1d-69f7f281c323\") "
Nov 02 21:41:00.882707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882573    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:00.882707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:00.882707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882649    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:00.882843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.882702    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:00.883191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.883159    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.883191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.883186    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.883372 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.883202    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.883372 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.883217    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.883372 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.883230    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.884520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.884488    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:00.884834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.884810    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:00.885079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.885056    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "kube-api-access-vsl6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:00.885157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.885087    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token" pod "c148f0bf-fd9c-4803-8c1d-69f7f281c323" (UID: "c148f0bf-fd9c-4803-8c1d-69f7f281c323"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:00.984674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.984627    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.984674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.984658    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsl6c\" (UniqueName: \"kubernetes.io/projected/c148f0bf-fd9c-4803-8c1d-69f7f281c323-kube-api-access-vsl6c\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.984674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.984669    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:00.984674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:00.984679    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c148f0bf-fd9c-4803-8c1d-69f7f281c323-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:01.474287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.474254    4888 generic.go:356] "Generic (PLEG): container finished" podID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerID="c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342" exitCode=0
Nov 02 21:41:01.474447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.474313    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"
Nov 02 21:41:01.474447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.474336    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerDied","Data":"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"}
Nov 02 21:41:01.474447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.474390    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh" event={"ID":"c148f0bf-fd9c-4803-8c1d-69f7f281c323","Type":"ContainerDied","Data":"211510ad2be1ad4f18d9ed0accf4dab699d92ae24785017804acf840203306be"}
Nov 02 21:41:01.474447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.474410    4888 scope.go:122] "RemoveContainer" containerID="c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"
Nov 02 21:41:01.486369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.486350    4888 scope.go:122] "RemoveContainer" containerID="08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"
Nov 02 21:41:01.495875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.495836    4888 scope.go:122] "RemoveContainer" containerID="c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"
Nov 02 21:41:01.496191 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:01.496171    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342\": container with ID starting with c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342 not found: ID does not exist" containerID="c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"
Nov 02 21:41:01.496278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.496208    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342"} err="failed to get container status \"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342\": rpc error: code = NotFound desc = could not find container \"c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342\": container with ID starting with c0971e7e39e7010aec7965827ab47a92699865e07195f70bcaf0f8a9c0c93342 not found: ID does not exist"
Nov 02 21:41:01.496278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.496237    4888 scope.go:122] "RemoveContainer" containerID="08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"
Nov 02 21:41:01.496530 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:01.496508    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af\": container with ID starting with 08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af not found: ID does not exist" containerID="08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"
Nov 02 21:41:01.496566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.496540    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af"} err="failed to get container status \"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af\": rpc error: code = NotFound desc = could not find container \"08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af\": container with ID starting with 08cbae0935bd76ae1e52ebe4bbe28aa9b6f12df10e0e402342c3c0fddb9e75af not found: ID does not exist"
Nov 02 21:41:01.679898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.679867    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:41:01.683220 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:01.683194    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh"]
Nov 02 21:41:02.615478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.615431    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" path="/var/lib/kubelet/pods/c148f0bf-fd9c-4803-8c1d-69f7f281c323/volumes"
Nov 02 21:41:02.767672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.740946    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:41:02.775862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.775799    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:02.775862 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.775860    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:02.776166 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776149    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy"
Nov 02 21:41:02.776166 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776166    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy"
Nov 02 21:41:02.776290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776197    4888 cpu_manager.go:414] "containerMap: removing container" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.776290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776206    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.780900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776472    4888 cpu_manager.go:414] "containerMap: removing container" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.780900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.776491    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.832041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.806481    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:02.832041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.806516    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ac93a91-7851-42ea-b49e-6d4db406ee0c" containerName="istio-proxy"
Nov 02 21:41:02.832041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.806538    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.832041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.806557    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="48fc9342-0659-42f5-b0ed-5475149426da" containerName="istio-proxy"
Nov 02 21:41:02.902737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.895221    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:02.918564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:02.917975    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:41:03.010294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.007214    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-9f77b7\""
Nov 02 21:41:03.010294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.007994    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-9f77b7-dockercfg-5gxsv\""
Nov 02 21:41:03.116033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.115977    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:41:03.152912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.149996    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g79f\" (UniqueName: \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.153239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.153202    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.159770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.159735    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:03.159962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.159948    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:03.160306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.160292    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c148f0bf-fd9c-4803-8c1d-69f7f281c323" containerName="istio-proxy"
Nov 02 21:41:03.164911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.164884    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.168962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.168939    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:41:03.254435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.254401    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.254619 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.254453    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g79f\" (UniqueName: \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.254669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.254629    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-777k9\" (UniqueName: \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.254758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.254740    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.255439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.255415    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.308798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.308769    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g79f\" (UniqueName: \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\") pod \"payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.356008 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.355955    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.356199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.356145    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-777k9\" (UniqueName: \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.356802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.356778    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.410719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.410633    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-777k9\" (UniqueName: \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\") pod \"payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.477758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.477720    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:03.495301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.495271    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:03.858001 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:03.857965    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a23439f_3c9e_4c05_9a78_39b2f7c170c0.slice/crio-f01e8d04ec6ae9d262bd92f03b80763b81a8ede92d0be7e3708dc48e670d9060 WatchSource:0}: Error finding container f01e8d04ec6ae9d262bd92f03b80763b81a8ede92d0be7e3708dc48e670d9060: Status 404 returned error can't find the container with id f01e8d04ec6ae9d262bd92f03b80763b81a8ede92d0be7e3708dc48e670d9060
Nov 02 21:41:03.858943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.858914    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:41:03.903527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:03.903500    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:41:03.904307 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:03.904276    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4f2d69_9649_47f3_beff_abc41b99330b.slice/crio-c500fdbd5a9e64e3bf09680d7bf6d74421f0ff56ad4af885115d1c38b7e98805 WatchSource:0}: Error finding container c500fdbd5a9e64e3bf09680d7bf6d74421f0ff56ad4af885115d1c38b7e98805: Status 404 returned error can't find the container with id c500fdbd5a9e64e3bf09680d7bf6d74421f0ff56ad4af885115d1c38b7e98805
Nov 02 21:41:04.519923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.519882    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" event={"ID":"3f4f2d69-9649-47f3-beff-abc41b99330b","Type":"ContainerStarted","Data":"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"}
Nov 02 21:41:04.520107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.519948    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" event={"ID":"3f4f2d69-9649-47f3-beff-abc41b99330b","Type":"ContainerStarted","Data":"c500fdbd5a9e64e3bf09680d7bf6d74421f0ff56ad4af885115d1c38b7e98805"}
Nov 02 21:41:04.520107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.519982    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:04.521607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.521581    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" event={"ID":"2a23439f-3c9e-4c05-9a78-39b2f7c170c0","Type":"ContainerStarted","Data":"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"}
Nov 02 21:41:04.521726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.521626    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" event={"ID":"2a23439f-3c9e-4c05-9a78-39b2f7c170c0","Type":"ContainerStarted","Data":"f01e8d04ec6ae9d262bd92f03b80763b81a8ede92d0be7e3708dc48e670d9060"}
Nov 02 21:41:04.521726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.521670    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:04.633057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.633006    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" podStartSLOduration=2.632990101 podStartE2EDuration="2.632990101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:04.62973385 +0000 UTC m=+1694.173525767" watchObservedRunningTime="2026-11-02 21:41:04.632990101 +0000 UTC m=+1694.176782014"
Nov 02 21:41:04.738642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:04.738577    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" podStartSLOduration=2.738558432 podStartE2EDuration="2.738558432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:04.736605491 +0000 UTC m=+1694.280397408" watchObservedRunningTime="2026-11-02 21:41:04.738558432 +0000 UTC m=+1694.282350350"
Nov 02 21:41:05.924284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:05.924240    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:41:05.924881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:05.924497    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" containerID="cri-o://b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36" gracePeriod=30
Nov 02 21:41:07.401871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.395356    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:41:07.412818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.411549    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.471983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.471950    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-d4d045-dockercfg-g6spn\""
Nov 02 21:41:07.472435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.472414    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-d4d045\""
Nov 02 21:41:07.583869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.578008    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:41:07.609236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.608885    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.609236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.609013    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7465m\" (UniqueName: \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.611450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.611419    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:41:07.689875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.665073    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.720874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.707628    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:41:07.720874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.720074    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.720874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.720253    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rgdc\" (UniqueName: \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.720874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.720377    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.720874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.720547    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7465m\" (UniqueName: \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.790873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.789935    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.864956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.829058    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.864956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.829202    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rgdc\" (UniqueName: \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.864956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.831298    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:07.864956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.858218    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7465m\" (UniqueName: \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\") pod \"payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:07.883981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:07.883949    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rgdc\" (UniqueName: \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\") pod \"payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:08.008023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.007943    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:08.098280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.098248    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:08.385904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.385878    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:41:08.386036 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:08.386013    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6885c5c_256b_4432_b4a2_cfbe5618d2a7.slice/crio-c6ca72cfa5cfc8c6d5294e6e2eacb3af55e23ca5952905ee90e4caff4d377909 WatchSource:0}: Error finding container c6ca72cfa5cfc8c6d5294e6e2eacb3af55e23ca5952905ee90e4caff4d377909: Status 404 returned error can't find the container with id c6ca72cfa5cfc8c6d5294e6e2eacb3af55e23ca5952905ee90e4caff4d377909
Nov 02 21:41:08.487434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.487402    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:41:08.487754 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:08.487642    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7956a661_a777_4747_9716_80595f3b2634.slice/crio-c0cd327ec221050ed0f54562394c4d10253aeeea77535923c8f895bc8631a3bf WatchSource:0}: Error finding container c0cd327ec221050ed0f54562394c4d10253aeeea77535923c8f895bc8631a3bf: Status 404 returned error can't find the container with id c0cd327ec221050ed0f54562394c4d10253aeeea77535923c8f895bc8631a3bf
Nov 02 21:41:08.588587 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.588551    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" event={"ID":"7956a661-a777-4747-9716-80595f3b2634","Type":"ContainerStarted","Data":"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"}
Nov 02 21:41:08.588759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.588608    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" event={"ID":"7956a661-a777-4747-9716-80595f3b2634","Type":"ContainerStarted","Data":"c0cd327ec221050ed0f54562394c4d10253aeeea77535923c8f895bc8631a3bf"}
Nov 02 21:41:08.588759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.588648    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:08.590691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.590663    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" event={"ID":"b6885c5c-256b-4432-b4a2-cfbe5618d2a7","Type":"ContainerStarted","Data":"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"}
Nov 02 21:41:08.590804 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.590718    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:08.590804 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.590737    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" event={"ID":"b6885c5c-256b-4432-b4a2-cfbe5618d2a7","Type":"ContainerStarted","Data":"c6ca72cfa5cfc8c6d5294e6e2eacb3af55e23ca5952905ee90e4caff4d377909"}
Nov 02 21:41:08.699743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.699685    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" podStartSLOduration=1.699669769 podStartE2EDuration="1.699669769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:08.696171413 +0000 UTC m=+1698.239963330" watchObservedRunningTime="2026-11-02 21:41:08.699669769 +0000 UTC m=+1698.243461688"
Nov 02 21:41:08.810505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:08.810436    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" podStartSLOduration=1.810413693 podStartE2EDuration="1.810413693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:08.803999241 +0000 UTC m=+1698.347791158" watchObservedRunningTime="2026-11-02 21:41:08.810413693 +0000 UTC m=+1698.354205612"
Nov 02 21:41:10.213779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:10.213737    4888 patch_prober.go:29] interesting pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:41:10.214195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:10.213803    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:41:11.382134 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.382099    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:41:11.392099 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392061    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392142    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392197    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392267    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392298    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392333    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392416    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:11.392568 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392544    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392624 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392611    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.392680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.392665    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\") pod \"7957c066-92a0-48d7-944f-663a277e87f9\" (UID: \"7957c066-92a0-48d7-944f-663a277e87f9\") "
Nov 02 21:41:11.393155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393122    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:11.393293 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393255    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:11.393390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393365    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:11.393505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393485    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:11.393566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393550    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7957c066-92a0-48d7-944f-663a277e87f9-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.393622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393570    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.393622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393586    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.393622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.393599    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.395683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.395637    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:11.395812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.395773    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:11.396157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.396132    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:11.397349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.397327    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h" pod "7957c066-92a0-48d7-944f-663a277e87f9" (UID: "7957c066-92a0-48d7-944f-663a277e87f9"). InnerVolumeSpecName "kube-api-access-l4z4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:11.494662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.494627    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7957c066-92a0-48d7-944f-663a277e87f9-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.494662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.494659    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.494662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.494668    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7957c066-92a0-48d7-944f-663a277e87f9-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.494925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.494677    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.494925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.494686    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4z4h\" (UniqueName: \"kubernetes.io/projected/7957c066-92a0-48d7-944f-663a277e87f9-kube-api-access-l4z4h\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:11.612816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.612727    4888 generic.go:356] "Generic (PLEG): container finished" podID="7957c066-92a0-48d7-944f-663a277e87f9" containerID="b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36" exitCode=0
Nov 02 21:41:11.612816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.612778    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"
Nov 02 21:41:11.612816 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.612792    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" event={"ID":"7957c066-92a0-48d7-944f-663a277e87f9","Type":"ContainerDied","Data":"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"}
Nov 02 21:41:11.613092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.612835    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht" event={"ID":"7957c066-92a0-48d7-944f-663a277e87f9","Type":"ContainerDied","Data":"ce2dc6cefbd37d3f525fdae9d562e8cc05baea28b64a92671dcf40bf87d617be"}
Nov 02 21:41:11.613092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.612869    4888 scope.go:122] "RemoveContainer" containerID="b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"
Nov 02 21:41:11.625033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.625012    4888 scope.go:122] "RemoveContainer" containerID="b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"
Nov 02 21:41:11.625355 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:11.625335    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36\": container with ID starting with b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36 not found: ID does not exist" containerID="b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"
Nov 02 21:41:11.625420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.625364    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36"} err="failed to get container status \"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36\": rpc error: code = NotFound desc = could not find container \"b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36\": container with ID starting with b7fdc3ef21c148052affd1c13a39f0f4fa4b328ff5ef59797286f862b10d2d36 not found: ID does not exist"
Nov 02 21:41:11.771562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.771521    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:41:11.786223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:11.786186    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht"]
Nov 02 21:41:12.603191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:12.603157    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7957c066-92a0-48d7-944f-663a277e87f9" path="/var/lib/kubelet/pods/7957c066-92a0-48d7-944f-663a277e87f9/volumes"
Nov 02 21:41:15.559877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:15.558094    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:41:15.575237 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:15.575193    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:41:19.600002 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:19.599971    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:41:19.601228 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:19.600503    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:41:20.387169 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:20.387129    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:41:20.387371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:20.387347    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy" containerID="cri-o://b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0" gracePeriod=30
Nov 02 21:41:21.586586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.586538    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:41:21.587185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.586772    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" containerID="cri-o://5fd06cb962cd286c4c741bf8452444b6e14d481eb9a568437cfb522872082024" gracePeriod=30
Nov 02 21:41:21.644058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.644013    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:41:21.644314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.644267    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy" containerID="cri-o://e0fd34a94fe62e4245f7e87fb23f16aceeae76a57ca56f1f7a4205436f3b4dd0" gracePeriod=30
Nov 02 21:41:21.652276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.652245    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:41:21.652500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:21.652475    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy" containerID="cri-o://cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768" gracePeriod=30
Nov 02 21:41:22.266949 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:22.266908    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:41:25.700550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.700520    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:41:25.744044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.744010    4888 generic.go:356] "Generic (PLEG): container finished" podID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerID="8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9" exitCode=0
Nov 02 21:41:25.744216 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.744089    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerDied","Data":"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"}
Nov 02 21:41:25.744587 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.744570    4888 scope.go:122] "RemoveContainer" containerID="8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"
Nov 02 21:41:25.745655 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.745631    4888 generic.go:356] "Generic (PLEG): container finished" podID="03e162ea-0824-4415-b003-b10224a62149" containerID="b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0" exitCode=0
Nov 02 21:41:25.745762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.745742    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"
Nov 02 21:41:25.745828 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.745795    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" event={"ID":"03e162ea-0824-4415-b003-b10224a62149","Type":"ContainerDied","Data":"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"}
Nov 02 21:41:25.745917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.745902    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc" event={"ID":"03e162ea-0824-4415-b003-b10224a62149","Type":"ContainerDied","Data":"9f7def421ed1dfe5c113fdcdf7c7529fe4fa48f9fb184ed10466485476aa2785"}
Nov 02 21:41:25.745968 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.745939    4888 scope.go:122] "RemoveContainer" containerID="b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"
Nov 02 21:41:25.746382 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.746347    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:25.746474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.746426    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:25.746474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.746465    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:25.765577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.765521    4888 scope.go:122] "RemoveContainer" containerID="b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"
Nov 02 21:41:25.767017 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:25.766990    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0\": container with ID starting with b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0 not found: ID does not exist" containerID="b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"
Nov 02 21:41:25.767129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.767025    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0"} err="failed to get container status \"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0\": rpc error: code = NotFound desc = could not find container \"b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0\": container with ID starting with b68483858674c8f6c8a2493a7ba456f464f630e71fd63cb4a9c57c0fb3b952c0 not found: ID does not exist"
Nov 02 21:41:25.788579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788544    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.788754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788594    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.788754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788634    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.788754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788688    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.788754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788710    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.788754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788737    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.789028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788878    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.789028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788921    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.789028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.788943    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\") pod \"03e162ea-0824-4415-b003-b10224a62149\" (UID: \"03e162ea-0824-4415-b003-b10224a62149\") "
Nov 02 21:41:25.790416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.790078    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:25.790416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.790119    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:25.790416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.790166    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:25.790416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.790390    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:25.792207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.792146    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:25.792400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.792370    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:25.794301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.794272    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:25.794468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.794444    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "kube-api-access-5jdrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:25.794536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.794515    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token" pod "03e162ea-0824-4415-b003-b10224a62149" (UID: "03e162ea-0824-4415-b003-b10224a62149"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:25.890909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890872    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.890909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890909    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890924    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e162ea-0824-4415-b003-b10224a62149-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890938    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e162ea-0824-4415-b003-b10224a62149-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890953    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jdrs\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-kube-api-access-5jdrs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890969    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890982    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e162ea-0824-4415-b003-b10224a62149-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.890996    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:25.891180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:25.891011    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e162ea-0824-4415-b003-b10224a62149-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:26.170248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.170205    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:41:26.175090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.174489    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc"]
Nov 02 21:41:26.274344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.274310    4888 patch_prober.go:29] interesting pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:41:26.274541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.274387    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:41:26.605789 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.605752    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03e162ea-0824-4415-b003-b10224a62149" path="/var/lib/kubelet/pods/03e162ea-0824-4415-b003-b10224a62149/volumes"
Nov 02 21:41:26.764798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.756643    4888 generic.go:356] "Generic (PLEG): container finished" podID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerID="e0fd34a94fe62e4245f7e87fb23f16aceeae76a57ca56f1f7a4205436f3b4dd0" exitCode=0
Nov 02 21:41:26.764798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.756713    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" event={"ID":"64c627dd-d460-4e2f-8e73-8dbf3d391aec","Type":"ContainerDied","Data":"e0fd34a94fe62e4245f7e87fb23f16aceeae76a57ca56f1f7a4205436f3b4dd0"}
Nov 02 21:41:26.764798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.759918    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerStarted","Data":"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"}
Nov 02 21:41:26.764798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.763582    4888 generic.go:356] "Generic (PLEG): container finished" podID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerID="5fd06cb962cd286c4c741bf8452444b6e14d481eb9a568437cfb522872082024" exitCode=0
Nov 02 21:41:26.764798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.763715    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" event={"ID":"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d","Type":"ContainerDied","Data":"5fd06cb962cd286c4c741bf8452444b6e14d481eb9a568437cfb522872082024"}
Nov 02 21:41:26.972246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:26.972219    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:41:27.054772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.054744    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055137    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055297    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055550 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055540    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055631    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055758    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.055903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.055887    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.056058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.056043    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.057087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.057066    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\") pod \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\" (UID: \"64c627dd-d460-4e2f-8e73-8dbf3d391aec\") "
Nov 02 21:41:27.058104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.056176    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.058230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.057067    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.058337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.057346    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:27.059004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.058956    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.062048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.062020    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.062308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.062263    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.064474 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.064434    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "kube-api-access-h4qdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.066427 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.066406    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.069308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.069194    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo" pod "64c627dd-d460-4e2f-8e73-8dbf3d391aec" (UID: "64c627dd-d460-4e2f-8e73-8dbf3d391aec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:27.072800 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.072780    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:41:27.125870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.124269    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:41:27.158607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158565    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.158812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158644    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.158812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158677    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.158974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158815    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.158974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158904    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.159094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.158994    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.159150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159111    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.159206 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159166    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.159260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159237    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\") pod \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\" (UID: \"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d\") "
Nov 02 21:41:27.159916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159890    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159922    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159940    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159952    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159965    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159978    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.159993    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4qdm\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-kube-api-access-h4qdm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160010    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c627dd-d460-4e2f-8e73-8dbf3d391aec-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160023    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c627dd-d460-4e2f-8e73-8dbf3d391aec-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.160583 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160557    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.160668 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160585    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.160915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160890    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.161016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160894    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:27.161016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.160943    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.161576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.161550    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.162179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.162157    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:27.162528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.162508    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "kube-api-access-6l4wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.163450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.163428    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy" pod "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" (UID: "1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.207831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.207745    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:41:27.208891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.208835    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:27.209028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.208936    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:27.260806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.260768    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261043    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.261156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261142    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261445 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261427    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261465    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261531    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261623    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.261766 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261746    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.261883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.261865    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.262030 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262013    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.262092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262058    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\") pod \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\" (UID: \"1fddb0e6-dcad-40fd-a480-a8be9ceb3874\") "
Nov 02 21:41:27.262170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262151    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.262469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262447    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:41:27.262650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.262886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262842    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262894    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262910    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262925    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262939    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262954    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.262972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262969    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262982    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.262996    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263009    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263022    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263037    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263050    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263066    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l4wd\" (UniqueName: \"kubernetes.io/projected/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d-kube-api-access-6l4wd\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.263692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.263674    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:41:27.264902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.264876    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:41:27.265406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.265377    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "kube-api-access-kvnvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.266299 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.266277    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token" pod "1fddb0e6-dcad-40fd-a480-a8be9ceb3874" (UID: "1fddb0e6-dcad-40fd-a480-a8be9ceb3874"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:27.368181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.364175    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvnvk\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-kube-api-access-kvnvk\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.368181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.364205    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.368181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.364216    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.368181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.364229    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fddb0e6-dcad-40fd-a480-a8be9ceb3874-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:41:27.816883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.816826    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq" event={"ID":"1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d","Type":"ContainerDied","Data":"2ae2b7763cba6dbc593ce54d9f14fb74c69b520cc946c74a35dd16e1a0cd6c25"}
Nov 02 21:41:27.817459 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.817436    4888 scope.go:122] "RemoveContainer" containerID="5fd06cb962cd286c4c741bf8452444b6e14d481eb9a568437cfb522872082024"
Nov 02 21:41:27.817759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.817741    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"
Nov 02 21:41:27.833146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.833112    4888 generic.go:356] "Generic (PLEG): container finished" podID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerID="cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768" exitCode=0
Nov 02 21:41:27.833480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.833361    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"
Nov 02 21:41:27.842554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.833380    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" event={"ID":"1fddb0e6-dcad-40fd-a480-a8be9ceb3874","Type":"ContainerDied","Data":"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"}
Nov 02 21:41:27.842554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.834377    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd" event={"ID":"1fddb0e6-dcad-40fd-a480-a8be9ceb3874","Type":"ContainerDied","Data":"9e53b1e976e2df1590c02405b60e330dbb93e9697bbb125acfcbcecda8ee8f44"}
Nov 02 21:41:27.919212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.917806    4888 generic.go:356] "Generic (PLEG): container finished" podID="d36e86b4-ae71-4466-82c8-6412967f2836" containerID="f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7" exitCode=0
Nov 02 21:41:27.919212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.918055    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerDied","Data":"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"}
Nov 02 21:41:27.919212 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.918592    4888 scope.go:122] "RemoveContainer" containerID="f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"
Nov 02 21:41:27.955910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.952066    4888 scope.go:122] "RemoveContainer" containerID="cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"
Nov 02 21:41:27.955910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.953614    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:27.955910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.953720    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:27.955910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.953768    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:27.960872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.957469    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"
Nov 02 21:41:27.985409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:27.957891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz" event={"ID":"64c627dd-d460-4e2f-8e73-8dbf3d391aec","Type":"ContainerDied","Data":"aa82125ffa4b9e4f3d86ca7acba04f5b7084d75c914701e806cc56140aef3c19"}
Nov 02 21:41:28.053104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.053086    4888 scope.go:122] "RemoveContainer" containerID="cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"
Nov 02 21:41:28.053424 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:28.053405    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768\": container with ID starting with cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768 not found: ID does not exist" containerID="cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"
Nov 02 21:41:28.053497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.053440    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768"} err="failed to get container status \"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768\": rpc error: code = NotFound desc = could not find container \"cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768\": container with ID starting with cacc66ffdc204aa344156ef6cd9cdb997da96657c9f11bb69f983653e4ede768 not found: ID does not exist"
Nov 02 21:41:28.053497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.053466    4888 scope.go:122] "RemoveContainer" containerID="e0fd34a94fe62e4245f7e87fb23f16aceeae76a57ca56f1f7a4205436f3b4dd0"
Nov 02 21:41:28.229874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.225970    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:28.229874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.226026    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:28.241879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.241820    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:41:28.242118 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.242102    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq"]
Nov 02 21:41:28.347923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.345690    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:41:28.349316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.349286    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz"]
Nov 02 21:41:28.456895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.456768    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:41:28.457618 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.457592    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd"]
Nov 02 21:41:28.478839 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.478730    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:41:28.652442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.652362    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" path="/var/lib/kubelet/pods/1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d/volumes"
Nov 02 21:41:28.668703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.668668    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" path="/var/lib/kubelet/pods/1fddb0e6-dcad-40fd-a480-a8be9ceb3874/volumes"
Nov 02 21:41:28.669680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.669656    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" path="/var/lib/kubelet/pods/64c627dd-d460-4e2f-8e73-8dbf3d391aec/volumes"
Nov 02 21:41:28.974877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:28.974815    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerStarted","Data":"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"}
Nov 02 21:41:29.008092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:29.008062    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:41:29.009104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:29.009083    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:29.009210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:29.009142    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:29.207394 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:29.207361    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:29.207574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:29.207417    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:30.008087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:30.008046    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:30.008469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:30.008111    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:30.207690 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:30.207653    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:30.207872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:30.207727    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:30.513876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:30.506023    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:41:31.008521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:31.008483    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:31.008969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:31.008546    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:31.208317 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:31.208284    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:31.208487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:31.208336    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:32.007990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:32.007950    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:32.008147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:32.008013    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:32.207590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:32.207556    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 02 21:41:32.208015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:32.207613    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 02 21:41:33.008164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.008128    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:33.008379 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.008193    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:33.238880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.238474    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:41:33.252902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.248209    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 21:41:33.705912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.705876    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:41:33.706579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706560    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706581    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706611    4888 cpu_manager.go:414] "containerMap: removing container" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706619    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706638    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706645    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706655    4888 cpu_manager.go:414] "containerMap: removing container" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy"
Nov 02 21:41:33.706684 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706663    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706691    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706699    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706871    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fddb0e6-dcad-40fd-a480-a8be9ceb3874" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706889    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="64c627dd-d460-4e2f-8e73-8dbf3d391aec" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706901    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f23da92-5f6a-40ed-9ea1-bfc24b8e2d0d" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706910    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="03e162ea-0824-4415-b003-b10224a62149" containerName="istio-proxy"
Nov 02 21:41:33.707071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.706924    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7957c066-92a0-48d7-944f-663a277e87f9" containerName="istio-proxy"
Nov 02 21:41:33.720323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.720288    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.762899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.762690    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:41:33.770238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.770199    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d3b375-serving-cert\""
Nov 02 21:41:33.870956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.870920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.871161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.871135    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.871305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.871278    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.871346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.871323    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2gj\" (UniqueName: \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.976278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.976177    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.977504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.977390    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.977746 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.977726    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2gj\" (UniqueName: \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.978055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.978030    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.978450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:33.978426    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:33.987197 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:33.987160    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-serving-cert: secret "maas-api-e2e-api-a-d3b375-serving-cert" not found
Nov 02 21:41:33.987519 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:33.987495    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls podName:3a22d739-3bf0-448b-9eb2-1e607ec5668c nodeName:}" failed. No retries permitted until 2026-11-02 21:41:34.487466647 +0000 UTC m=+1724.031258569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls") pod "maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" (UID: "3a22d739-3bf0-448b-9eb2-1e607ec5668c") : secret "maas-api-e2e-api-a-d3b375-serving-cert" not found
Nov 02 21:41:34.008292 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.008252    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 02 21:41:34.009951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.009913    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 02 21:41:34.031868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.030828    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:34.065421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.065384    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2gj\" (UniqueName: \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:34.587622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.587580    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:34.590180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.590154    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") pod \"maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:34.645071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:34.645035    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:35.013581 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.013538    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:41:35.013978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.013955    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:41:35.014711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.014693    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:41:35.034998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.034964    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:41:35.035309 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:35.035285    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a22d739_3bf0_448b_9eb2_1e607ec5668c.slice/crio-4a62bbaddfe863c151b87b1b2e3e141eddd55ce9c5e2466c56fc02f151cb72ff WatchSource:0}: Error finding container 4a62bbaddfe863c151b87b1b2e3e141eddd55ce9c5e2466c56fc02f151cb72ff: Status 404 returned error can't find the container with id 4a62bbaddfe863c151b87b1b2e3e141eddd55ce9c5e2466c56fc02f151cb72ff
Nov 02 21:41:35.037407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.037383    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:41:35.060475 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:35.060411    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" event={"ID":"3a22d739-3bf0-448b-9eb2-1e607ec5668c","Type":"ContainerStarted","Data":"4a62bbaddfe863c151b87b1b2e3e141eddd55ce9c5e2466c56fc02f151cb72ff"}
Nov 02 21:41:36.073531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:36.073458    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" event={"ID":"3a22d739-3bf0-448b-9eb2-1e607ec5668c","Type":"ContainerStarted","Data":"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"}
Nov 02 21:41:36.214756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:36.214692    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" podStartSLOduration=2.278266935 podStartE2EDuration="3.214677214s" totalImagesPullingTime="936.410279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:36.212389337 +0000 UTC m=+1725.756181254" watchObservedRunningTime="2026-11-02 21:41:36.214677214 +0000 UTC m=+1725.758469131"
Nov 02 21:41:37.080114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:37.080077    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:38.814524 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.814485    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:41:38.819074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.819052    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.873963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.873915    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:41:38.946797 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.946765    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6b983e02\"/\"openshift-service-ca.crt\""
Nov 02 21:41:38.947056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.946927    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6b983e02\"/\"default-dockercfg-rqd75\""
Nov 02 21:41:38.947056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.946983    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6b983e02\"/\"kube-root-ca.crt\""
Nov 02 21:41:38.947056 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.947042    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6b983e02\"/\"e2e-lifecycle-model-6b983e02-kserve-self-signed-certs\""
Nov 02 21:41:38.975678 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.975642    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.975890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.975713    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.975890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.975815    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.975890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.975878    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfgqn\" (UniqueName: \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.976020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.975982    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:38.976053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:38.976023    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.077996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.077903    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.078275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.078253    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.078418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.078397    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfgqn\" (UniqueName: \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.078631 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.078616    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.078788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.078774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.079045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.079025    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.079744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.079719    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.085869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.080152    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.087216 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.087174    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.101431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.101396    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.102520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.102494    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.151058 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.151021    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfgqn\" (UniqueName: \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\") pod \"e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") " pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.450875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.450435    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:39.900320 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:39.900284    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:41:39.900677 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:39.900442    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac5a6da6_b24e_44de_8ce5_2b68cc40a49d.slice/crio-59cd3d6dbabbd8aeb66762de3c0152ddec9eaed3609e7d1ed9ad75f9ff72eddb WatchSource:0}: Error finding container 59cd3d6dbabbd8aeb66762de3c0152ddec9eaed3609e7d1ed9ad75f9ff72eddb: Status 404 returned error can't find the container with id 59cd3d6dbabbd8aeb66762de3c0152ddec9eaed3609e7d1ed9ad75f9ff72eddb
Nov 02 21:41:40.131904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:40.131843    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" event={"ID":"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d","Type":"ContainerStarted","Data":"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"}
Nov 02 21:41:40.132090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:40.131914    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" event={"ID":"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d","Type":"ContainerStarted","Data":"59cd3d6dbabbd8aeb66762de3c0152ddec9eaed3609e7d1ed9ad75f9ff72eddb"}
Nov 02 21:41:40.241665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:40.241610    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podStartSLOduration=2.24159354 podStartE2EDuration="2.24159354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:40.240170775 +0000 UTC m=+1729.783962716" watchObservedRunningTime="2026-11-02 21:41:40.24159354 +0000 UTC m=+1729.785385456"
Nov 02 21:41:40.452049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:40.452011    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:43.102480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:43.102451    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:41:46.095834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.095797    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 21:41:46.100652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.100625    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.141707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.141655    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.141914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.141712    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.141914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.141742    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.141999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.141965    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.142043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.141994    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.142098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.142081    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.142152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.142140    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnlw\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.142194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.142161    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.142264 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.142252    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.150162 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.150133    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f22129-openshift-default-dockercfg-6q46c\""
Nov 02 21:41:46.168275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.168238    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.243768    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.243823    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.243909    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.243988    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnlw\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.244018    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.244091    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.244164    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.244202    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.244242    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.245020    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.247883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.246175    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.249634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.249605    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.266053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.250059    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.266335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.251730    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.374714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.374627    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.377720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.377689    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.378253 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.378230    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.403669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.403626    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnlw\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\") pod \"e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") " pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:46.422891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:46.422558    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:47.028871 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:47.028818    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b23d49a_ff76_4d09_9b0d_ea60fbf037cf.slice/crio-fde1ec28dee1269ec27cac34e3aa3d6d22c570764359fc7707eb73dae876b431 WatchSource:0}: Error finding container fde1ec28dee1269ec27cac34e3aa3d6d22c570764359fc7707eb73dae876b431: Status 404 returned error can't find the container with id fde1ec28dee1269ec27cac34e3aa3d6d22c570764359fc7707eb73dae876b431
Nov 02 21:41:47.030233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.030206    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 21:41:47.031835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.031800    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:47.031961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.031889    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:47.031961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.031921    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:41:47.377977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.377878    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerStarted","Data":"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"}
Nov 02 21:41:47.377977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.377951    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerStarted","Data":"fde1ec28dee1269ec27cac34e3aa3d6d22c570764359fc7707eb73dae876b431"}
Nov 02 21:41:47.422881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.422823    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:47.494446 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:47.494384    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podStartSLOduration=1.494367265 podStartE2EDuration="1.494367265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:47.489740571 +0000 UTC m=+1737.033532489" watchObservedRunningTime="2026-11-02 21:41:47.494367265 +0000 UTC m=+1737.038159227"
Nov 02 21:41:48.428391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:48.428361    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:49.392194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:49.392165    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:49.393336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:49.393310    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:41:49.451823 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:49.451790    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:49.464798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:49.464764    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:50.411421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:50.411391    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:41:51.331638 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:51.331591    4888 scope.go:122] "RemoveContainer" containerID="c637456e9e1bed4b1662aa9dc4addb5dcf857406a8f424352fe5df5953b78a21"
Nov 02 21:41:52.261243 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:41:52.261205    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:41:53.794037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:53.794001    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 21:41:54.173931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.131098    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 21:41:54.173931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.131254    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.208873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.205160    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f22129\""
Nov 02 21:41:54.254889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.254810    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f22129-dockercfg-rvh7q\""
Nov 02 21:41:54.256277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.256246    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.256457 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.256438    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28xbz\" (UniqueName: \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.270891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.270379    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 21:41:54.305536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.305248    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.358755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.358717    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.359333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.359302    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28xbz\" (UniqueName: \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.375453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.373559    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.407871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.405372    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 21:41:54.475111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.475020    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.475593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.475560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxl8j\" (UniqueName: \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.489205 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.489175    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28xbz\" (UniqueName: \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\") pod \"payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.577922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.577872    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxl8j\" (UniqueName: \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.578396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.578356    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.584803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.579355    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.634195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.634156    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxl8j\" (UniqueName: \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\") pod \"payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:54.682425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.682391    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:54.740936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:54.718199    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:55.284947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.284917    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 21:41:55.285450 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:55.285422    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2fc1de_91b7_4350_ac03_c63922a3af6c.slice/crio-f41d43c0bb29bb1d81c59325892b7831926890986b4821459b7383c8c0127b9b WatchSource:0}: Error finding container f41d43c0bb29bb1d81c59325892b7831926890986b4821459b7383c8c0127b9b: Status 404 returned error can't find the container with id f41d43c0bb29bb1d81c59325892b7831926890986b4821459b7383c8c0127b9b
Nov 02 21:41:55.389284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.389253    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 21:41:55.389396 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:41:55.389265    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1921f4a1_4fcf_4a87_9600_de8bd25ce0be.slice/crio-72fc30cb51882bab97ca38a40a6fed2ed414275086aeaf91f87a23ac0d049fc6 WatchSource:0}: Error finding container 72fc30cb51882bab97ca38a40a6fed2ed414275086aeaf91f87a23ac0d049fc6: Status 404 returned error can't find the container with id 72fc30cb51882bab97ca38a40a6fed2ed414275086aeaf91f87a23ac0d049fc6
Nov 02 21:41:55.592287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.592195    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" event={"ID":"7a2fc1de-91b7-4350-ac03-c63922a3af6c","Type":"ContainerStarted","Data":"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"}
Nov 02 21:41:55.592287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.592254    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" event={"ID":"7a2fc1de-91b7-4350-ac03-c63922a3af6c","Type":"ContainerStarted","Data":"f41d43c0bb29bb1d81c59325892b7831926890986b4821459b7383c8c0127b9b"}
Nov 02 21:41:55.592527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.592367    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:41:55.593829 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.593795    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" event={"ID":"1921f4a1-4fcf-4a87-9600-de8bd25ce0be","Type":"ContainerStarted","Data":"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"}
Nov 02 21:41:55.593963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.593836    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" event={"ID":"1921f4a1-4fcf-4a87-9600-de8bd25ce0be","Type":"ContainerStarted","Data":"72fc30cb51882bab97ca38a40a6fed2ed414275086aeaf91f87a23ac0d049fc6"}
Nov 02 21:41:55.593963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.593895    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:41:55.702264 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.702209    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" podStartSLOduration=2.70219446 podStartE2EDuration="2.70219446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:55.700717256 +0000 UTC m=+1745.244509172" watchObservedRunningTime="2026-11-02 21:41:55.70219446 +0000 UTC m=+1745.245986376"
Nov 02 21:41:55.835872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:41:55.835367    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" podStartSLOduration=2.835345852 podStartE2EDuration="2.835345852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:55.807370083 +0000 UTC m=+1745.351162001" watchObservedRunningTime="2026-11-02 21:41:55.835345852 +0000 UTC m=+1745.379137768"
Nov 02 21:42:04.693571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.693533    4888 generic.go:356] "Generic (PLEG): container finished" podID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerID="aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd" exitCode=0
Nov 02 21:42:04.694150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.693602    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerDied","Data":"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"}
Nov 02 21:42:04.694234 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.694215    4888 scope.go:122] "RemoveContainer" containerID="aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"
Nov 02 21:42:04.696102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.696076    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:04.696176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.696148    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:04.696215 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:04.696185    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:05.703414 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:05.703377    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerStarted","Data":"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"}
Nov 02 21:42:06.394239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:06.394200    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:42:06.394447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:06.394373    4888 patch_prober.go:29] interesting pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 21:42:06.394447 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:06.394428    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 21:42:06.604487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:06.604453    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 21:42:06.604675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:06.604536    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 21:42:07.393177 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:07.393138    4888 patch_prober.go:29] interesting pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 21:42:07.393597 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:07.393192    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 21:42:08.392970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:08.392935    4888 patch_prober.go:29] interesting pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 21:42:08.393161 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:08.392989    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 21:42:09.392754 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:09.392714    4888 patch_prober.go:29] interesting pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 21:42:09.393298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:09.392769    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 21:42:10.393207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:10.393171    4888 patch_prober.go:29] interesting pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 21:42:10.393738 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:10.393245    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 21:42:11.397093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:11.397062    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:42:11.397596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:11.397374    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:42:11.398128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:11.398109    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:42:22.261030 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:42:22.260992    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:42:30.280980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.280939    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:42:30.286936 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.286911    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308163    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308201    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308451 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308451 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308287    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308485    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308545    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308677    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308771 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308723    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdlvz\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.308897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.308879    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.337613 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.337556    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-f13b18-openshift-default-dockercfg-6vltr\""
Nov 02 21:42:30.390770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.390735    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:42:30.410155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410116    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410200    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410354 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410252    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdlvz\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410382    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410475    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410501    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410532    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410555    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.410720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.410653    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.411274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.411246    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.428869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.411641    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.428869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.411745    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.428869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.412400    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.428869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.412910    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.539921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.527521    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.539921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.536954    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdlvz\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.571869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.571351    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.582518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.582473    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\") pod \"e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") " pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:30.642449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:30.641156    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:31.153324 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:42:31.153288    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2448d982_164d_43fc_9ca1_01c160c65369.slice/crio-b174fbbb618cda27a0d19c3513aa30349f19aa7135b3c55519c83ae83380cb56 WatchSource:0}: Error finding container b174fbbb618cda27a0d19c3513aa30349f19aa7135b3c55519c83ae83380cb56: Status 404 returned error can't find the container with id b174fbbb618cda27a0d19c3513aa30349f19aa7135b3c55519c83ae83380cb56
Nov 02 21:42:31.153456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:31.153320    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:42:31.156711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:31.156670    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:31.156800 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:31.156759    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:31.156843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:31.156803    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:32.008408 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:32.008368    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerStarted","Data":"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"}
Nov 02 21:42:32.008982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:32.008431    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerStarted","Data":"b174fbbb618cda27a0d19c3513aa30349f19aa7135b3c55519c83ae83380cb56"}
Nov 02 21:42:32.123539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:32.123483    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podStartSLOduration=2.123465554 podStartE2EDuration="2.123465554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:42:32.119417378 +0000 UTC m=+1781.663209296" watchObservedRunningTime="2026-11-02 21:42:32.123465554 +0000 UTC m=+1781.667257470"
Nov 02 21:42:32.656889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:32.656181    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:32.708933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:32.708821    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:33.016537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:33.016445    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:33.017380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:33.017360    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:42:38.401093 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.401051    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:42:38.421539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.421498    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.499083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.495921    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:42:38.503115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.503084    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-f13b18\""
Nov 02 21:42:38.504040 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.504018    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-f13b18-dockercfg-lbkzp\""
Nov 02 21:42:38.519086 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.519046    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.519453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.519427    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghznm\" (UniqueName: \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.643876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.632936    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.643876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.633156    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghznm\" (UniqueName: \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.643876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.639915    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:38.921876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.899425    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:42:38.972186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.972153    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:38.982244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:38.982206    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghznm\" (UniqueName: \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\") pod \"payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:39.004090 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.003297    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:42:39.041527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.041491    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:39.154363 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.154270    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.154541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.154455    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgccc\" (UniqueName: \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.256486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.256398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.256486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.256482    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgccc\" (UniqueName: \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.257155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.257134    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.337999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.337963    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgccc\" (UniqueName: \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\") pod \"payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.361348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.361308    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:39.457700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.457554    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:42:39.785245 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:42:39.785210    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491b9c98_366b_4e89_b8c1_b2bb6444f3ba.slice/crio-5e3a2555cfdbdaa8fb73de832012fd27e6ee2f21d8093cc7ca66790c381a4c88 WatchSource:0}: Error finding container 5e3a2555cfdbdaa8fb73de832012fd27e6ee2f21d8093cc7ca66790c381a4c88: Status 404 returned error can't find the container with id 5e3a2555cfdbdaa8fb73de832012fd27e6ee2f21d8093cc7ca66790c381a4c88
Nov 02 21:42:39.786891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:39.786859    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:42:40.070156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.070048    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" event={"ID":"491b9c98-366b-4e89-b8c1-b2bb6444f3ba","Type":"ContainerStarted","Data":"32fa0e7ff9e28740ac8dd693537fd7bb315b26820c14071f6e091e04022ac08b"}
Nov 02 21:42:40.070156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.070122    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" event={"ID":"491b9c98-366b-4e89-b8c1-b2bb6444f3ba","Type":"ContainerStarted","Data":"5e3a2555cfdbdaa8fb73de832012fd27e6ee2f21d8093cc7ca66790c381a4c88"}
Nov 02 21:42:40.070369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.070286    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:40.071988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.071952    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" event={"ID":"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2","Type":"ContainerStarted","Data":"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"}
Nov 02 21:42:40.072138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.072009    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" event={"ID":"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2","Type":"ContainerStarted","Data":"9e7f6f2d21d9ab4818e7c67d1c500751a4b256aa0c3aef24dd04ca61627c7fe8"}
Nov 02 21:42:40.072138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.072040    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:40.179308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.179256    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" podStartSLOduration=2.179237866 podStartE2EDuration="2.179237866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:42:40.178616337 +0000 UTC m=+1789.722408254" watchObservedRunningTime="2026-11-02 21:42:40.179237866 +0000 UTC m=+1789.723029781"
Nov 02 21:42:40.288893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:40.288677    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" podStartSLOduration=2.288657545 podStartE2EDuration="2.288657545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:42:40.286559899 +0000 UTC m=+1789.830351816" watchObservedRunningTime="2026-11-02 21:42:40.288657545 +0000 UTC m=+1789.832449462"
Nov 02 21:42:45.501112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:45.501064    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:42:45.501557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:45.501144    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:42:48.162025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.161990    4888 generic.go:356] "Generic (PLEG): container finished" podID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerID="2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429" exitCode=0
Nov 02 21:42:48.162458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.162063    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerDied","Data":"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"}
Nov 02 21:42:48.162562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.162547    4888 scope.go:122] "RemoveContainer" containerID="2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"
Nov 02 21:42:48.164227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.164199    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:48.164297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.164268    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:48.164336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:48.164301    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:49.171080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:49.171046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerStarted","Data":"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"}
Nov 02 21:42:50.161827 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:50.161794    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:42:50.162073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:50.162051    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 21:42:50.162131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:50.162113    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 21:42:50.851175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:50.851139    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:42:50.853939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:50.853909    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:42:51.088509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:51.088475    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:42:51.089140 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:51.089114    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:42:51.174725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:51.174278    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 21:42:51.174725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:51.174342    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 21:42:52.164396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.164359    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 21:42:52.165055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.165002    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 21:42:52.262886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.261749    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:42:52.282890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.282376    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.416865 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.411722    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-32de96-openshift-default-dockercfg-nrcrw\""
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425126    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425204    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425289    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425357    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mjtj\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425387    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425429    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425479    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425545    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.425604    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.435884 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:42:52.433553    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:42:52.456870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.452255    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:42:52.538031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.537984    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.540708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.540674    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.541473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.541449    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583598    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583705    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mjtj\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583736    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583836    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.583937    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.539732    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.584692    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.560280    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.585572    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.593266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.592510    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.604664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.601239    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.612665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.612623    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.640742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.640709    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.645285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.645243    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mjtj\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\") pod \"e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") " pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:52.716202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:52.716113    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:53.162498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.162461    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 21:42:53.162673 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.162517    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 21:42:53.204220 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:42:53.204184    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa99491_e8a5_4328_8bf6_2f2dc5446954.slice/crio-fa02a9a3ed18597a986a3c1c64c4542b10902cf3b527d6627498d9c1fe5bd1b7 WatchSource:0}: Error finding container fa02a9a3ed18597a986a3c1c64c4542b10902cf3b527d6627498d9c1fe5bd1b7: Status 404 returned error can't find the container with id fa02a9a3ed18597a986a3c1c64c4542b10902cf3b527d6627498d9c1fe5bd1b7
Nov 02 21:42:53.205480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.205453    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:42:53.207467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.207435    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:53.207544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.207503    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:53.207544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.207533    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:53.251448 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:53.251413    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerStarted","Data":"fa02a9a3ed18597a986a3c1c64c4542b10902cf3b527d6627498d9c1fe5bd1b7"}
Nov 02 21:42:54.162512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.162480    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 21:42:54.162693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.162532    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 21:42:54.260689 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.260657    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerStarted","Data":"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"}
Nov 02 21:42:54.386988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.386936    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podStartSLOduration=2.386921729 podStartE2EDuration="2.386921729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:42:54.384952819 +0000 UTC m=+1803.928744737" watchObservedRunningTime="2026-11-02 21:42:54.386921729 +0000 UTC m=+1803.930713646"
Nov 02 21:42:54.717860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.717790    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:54.724863 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:54.724813    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:55.166590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:55.166556    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:42:55.167102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:55.167081    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:42:55.168060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:55.168037    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:42:55.268155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:55.268119    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:55.269079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:55.269059    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:42:59.058586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.058553    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:42:59.063681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.063651    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.113677 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.113644    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a1bf7d-openshift-default-dockercfg-frtrm\""
Nov 02 21:42:59.145482 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145445    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.145685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145512    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.145685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145658    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.145788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145771    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.145838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145807    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4m2p\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.145932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145842    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.146005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.145985    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.146077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.146059    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.146139 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.146092    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.167465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.167415    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:42:59.261683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.261634    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.262219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.262152    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.262461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.262443    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.262692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.262677    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.262842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.262825    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4m2p\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.263197 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.263176    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.263440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.263424    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.263602 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.263578    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.263731 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.263712    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.268359 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.267907    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.269516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.269318    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.269516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.269480    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.279564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.279379    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.297087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.297042    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.307611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.307507    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.335391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.332214    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.335391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.332920    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4m2p\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.347541 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.345262    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\") pod \"e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") " pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.390147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.390110    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:42:59.885528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.885488    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:42:59.922552 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:42:59.922509    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150eec8f_a279_4e86_89ee_d9db889d304f.slice/crio-d75c9bced9c95c4a51d6b47f811ba756acaa6f7af31afc959d4054d23e10b0e1 WatchSource:0}: Error finding container d75c9bced9c95c4a51d6b47f811ba756acaa6f7af31afc959d4054d23e10b0e1: Status 404 returned error can't find the container with id d75c9bced9c95c4a51d6b47f811ba756acaa6f7af31afc959d4054d23e10b0e1
Nov 02 21:42:59.925582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.925547    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:59.925669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.925632    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:42:59.925669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:42:59.925665    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:00.371059 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:00.371019    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerStarted","Data":"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"}
Nov 02 21:43:00.371558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:00.371088    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerStarted","Data":"d75c9bced9c95c4a51d6b47f811ba756acaa6f7af31afc959d4054d23e10b0e1"}
Nov 02 21:43:00.403929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:00.403891    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:43:01.411069 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:01.411027    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:43:01.525625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:01.525572    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podStartSLOduration=2.525557221 podStartE2EDuration="2.525557221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:00.791766277 +0000 UTC m=+1810.335558185" watchObservedRunningTime="2026-11-02 21:43:01.525557221 +0000 UTC m=+1811.069349138"
Nov 02 21:43:02.385672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:02.385641    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:43:02.386635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:02.386608    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:43:07.915595 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:07.915559    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:43:07.924199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:07.924173    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:07.930561 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:07.930524    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:43:08.031726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.031675    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.031953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.031926    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczjb\" (UniqueName: \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.039357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.039332    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a1bf7d-dockercfg-k2vd9\""
Nov 02 21:43:08.039357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.039332    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a1bf7d\""
Nov 02 21:43:08.133339 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.133292    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hczjb\" (UniqueName: \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.133533 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.133410    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.134214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.134194    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.187598 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.187517    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczjb\" (UniqueName: \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\") pod \"payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") " pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.316993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.316946    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:08.691036 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:43:08.690999    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f67e1a_38b8_4e78_8208_8110b7160b71.slice/crio-a2654e2d1b5f9a4371311c19b064901111f1d6849aef82a0e2c6e6fd9eb4ce0e WatchSource:0}: Error finding container a2654e2d1b5f9a4371311c19b064901111f1d6849aef82a0e2c6e6fd9eb4ce0e: Status 404 returned error can't find the container with id a2654e2d1b5f9a4371311c19b064901111f1d6849aef82a0e2c6e6fd9eb4ce0e
Nov 02 21:43:08.691503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:08.691478    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:43:09.563957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:09.563901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" event={"ID":"c8f67e1a-38b8-4e78-8208-8110b7160b71","Type":"ContainerStarted","Data":"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"}
Nov 02 21:43:09.564465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:09.564010    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" event={"ID":"c8f67e1a-38b8-4e78-8208-8110b7160b71","Type":"ContainerStarted","Data":"a2654e2d1b5f9a4371311c19b064901111f1d6849aef82a0e2c6e6fd9eb4ce0e"}
Nov 02 21:43:09.564465 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:09.564052    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:20.574306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:20.574275    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:43:20.693349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:20.693292    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" podStartSLOduration=13.693276704 podStartE2EDuration="13.693276704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:09.718572236 +0000 UTC m=+1819.262364151" watchObservedRunningTime="2026-11-02 21:43:20.693276704 +0000 UTC m=+1830.237068619"
Nov 02 21:43:22.267127 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:43:22.267090    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:43:30.739186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.739148    4888 generic.go:356] "Generic (PLEG): container finished" podID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerID="34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff" exitCode=0
Nov 02 21:43:30.739646 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.739216    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerDied","Data":"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"}
Nov 02 21:43:30.739695 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.739665    4888 scope.go:122] "RemoveContainer" containerID="34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"
Nov 02 21:43:30.741452 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.741425    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:30.741521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.741492    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:30.741559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:30.741525    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:31.753472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:31.753435    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerStarted","Data":"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"}
Nov 02 21:43:32.426178 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:32.426142    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:43:32.440946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:32.434996    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:32.440946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:32.435063    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:33.249031 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.248983    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:43:33.254126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.254085    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.377878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.370568    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-5451a5-openshift-default-dockercfg-7jfx2\""
Nov 02 21:43:33.389535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.389465    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:43:33.398576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.322835    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.398765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.398641    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.398765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.398724    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrq4\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.398912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.398835    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.399013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.398991    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.399709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.399145    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.399709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.399225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.399709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.399341    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.399709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.399428    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.427534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.427495    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:33.427825 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.427791    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:33.503123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.502977    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.503424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.503392    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrq4\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.511914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.503585    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.513562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.513530    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.513880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.513838    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.521275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.521235    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.521571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.521549    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.521755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.521739    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.536871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.522111    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.537358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.513017    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.537555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.524712    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.537689 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.524958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.537807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.525003    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.550018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.525167    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.619895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.618898    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.641889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.621433    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.641889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.637236    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.670804 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.670768    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrq4\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:33.703182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:33.703150    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:34.337612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.337567    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:43:34.342978 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:43:34.340138    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84744649_e77a_47bd_a63d_eddfb0961762.slice/crio-11095b7cb98ccb44aeb664a456b81fcd2ed33b82739fa91c01515429e1b3d667 WatchSource:0}: Error finding container 11095b7cb98ccb44aeb664a456b81fcd2ed33b82739fa91c01515429e1b3d667: Status 404 returned error can't find the container with id 11095b7cb98ccb44aeb664a456b81fcd2ed33b82739fa91c01515429e1b3d667
Nov 02 21:43:34.347426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.346973    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:34.347426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.347047    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:34.347426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.347078    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:43:34.392893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.392839    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:43:34.429877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.423978    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:34.429877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.424038    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:34.870556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.870516    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerStarted","Data":"13efd7e8d80505038f3c838edeae5327c62705abcb8a8459b4653c114fc3541b"}
Nov 02 21:43:34.870768 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.870589    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerStarted","Data":"11095b7cb98ccb44aeb664a456b81fcd2ed33b82739fa91c01515429e1b3d667"}
Nov 02 21:43:34.985221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:34.985153    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podStartSLOduration=1.985138412 podStartE2EDuration="1.985138412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:34.984141026 +0000 UTC m=+1844.527932943" watchObservedRunningTime="2026-11-02 21:43:34.985138412 +0000 UTC m=+1844.528930329"
Nov 02 21:43:35.423483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.423438    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:35.424047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.423503    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:35.704761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.704677    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:35.725199 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.725164    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:35.877953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.877919    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:35.878995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:35.878970    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:43:36.429873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:36.426320    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:36.429873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:36.426388    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:37.423722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:37.423686    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 02 21:43:37.423929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:37.423744    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 02 21:43:38.429110 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:38.429075    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:43:38.430497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:43:38.430475    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 21:43:52.261602 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:43:52.261558    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:44:22.263083 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:44:22.263041    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:44:52.266585 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:44:52.266539    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:44:56.309091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:56.309043    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:44:56.309554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:56.309120    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:44:57.717023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.716983    4888 generic.go:356] "Generic (PLEG): container finished" podID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerID="939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409" exitCode=0
Nov 02 21:44:57.717470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.717051    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerDied","Data":"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"}
Nov 02 21:44:57.717515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.717480    4888 scope.go:122] "RemoveContainer" containerID="939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"
Nov 02 21:44:57.719283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.719254    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:44:57.719378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.719325    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:44:57.719378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:57.719359    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:44:58.742527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:58.742493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerStarted","Data":"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"}
Nov 02 21:44:59.720528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:59.720313    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:44:59.720958 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:59.720815    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 02 21:44:59.721158 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:44:59.721134    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 02 21:45:00.190608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.190395    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:00.195897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.195865    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:00.196129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.195997    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.201313 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.201283    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.208367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.208339    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:00.227389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.227356    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.243489 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.243454    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:00.249549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.249517    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:00.249686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.249614    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.255196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.255174    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:00.255327 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.255301    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.260326 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260303    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wng96\" (UniqueName: \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.260594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260578    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.260699 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260689    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.260826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260654    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.260929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260910    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nwt9\" (UniqueName: \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.260991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.260944    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.261049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.261012    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzr7w\" (UniqueName: \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.306355 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.306301    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:00.355803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.335183    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j922b\""
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362176    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362252    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wng96\" (UniqueName: \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362298    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362344    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c26r2\" (UniqueName: \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362415    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4fj\" (UniqueName: \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362518    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362555    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362714    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szws5\" (UniqueName: \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362750    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nwt9\" (UniqueName: \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362777    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362814    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzr7w\" (UniqueName: \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.362989    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.364015    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.364419    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.364777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.364706    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.365705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.365347    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:00.369518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.369496    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:00.373664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.373644    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:00.378111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.378083    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:00.392526 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.392471    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:00.443582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.443474    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzr7w\" (UniqueName: \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.444071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.444043    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wng96\" (UniqueName: \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\") pod \"maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.444180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.444070    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nwt9\" (UniqueName: \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\") pod \"maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.464433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.464385    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4fj\" (UniqueName: \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.465236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.465207    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szws5\" (UniqueName: \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.465508 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.465488    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.465657 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.465641    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.465807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.465791    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.465964 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.465946    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c26r2\" (UniqueName: \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.466708 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.466678    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.467238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.466967    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.467238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.467045    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.538021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.537977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4fj\" (UniqueName: \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\") pod \"maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.538209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.537977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c26r2\" (UniqueName: \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\") pod \"maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.538300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.538254    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szws5\" (UniqueName: \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\") pod \"maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.546160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.546130    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:00.558715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.558685    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:00.567723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.567692    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:00.575945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.575753    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:00.586371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.586333    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:00.596791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.595245    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:00.743888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.735411    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 02 21:45:00.743888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.735476    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 02 21:45:00.974207 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:00.974156    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ad8fbe_d2de_485e_9fcc_0be5bb520c98.slice/crio-6ba7b135f65cd5993a53de6d67791f4e5a3113d05b7b652e681f6f9e0ada033f WatchSource:0}: Error finding container 6ba7b135f65cd5993a53de6d67791f4e5a3113d05b7b652e681f6f9e0ada033f: Status 404 returned error can't find the container with id 6ba7b135f65cd5993a53de6d67791f4e5a3113d05b7b652e681f6f9e0ada033f
Nov 02 21:45:00.974333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:00.974289    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:01.241211 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:01.241167    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a5688a3_9288_4a87_a7dd_40e031ea93ca.slice/crio-b9de02b662fd757c7b422265b4a4345bc63fd6a779e3e67f3ab0aa6b3c3933f3 WatchSource:0}: Error finding container b9de02b662fd757c7b422265b4a4345bc63fd6a779e3e67f3ab0aa6b3c3933f3: Status 404 returned error can't find the container with id b9de02b662fd757c7b422265b4a4345bc63fd6a779e3e67f3ab0aa6b3c3933f3
Nov 02 21:45:01.244344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.244309    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:01.279041 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:01.279008    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a25c28_24ba_4c89_88b8_d97935ba945d.slice/crio-8f5c822357e20c7a10f7ea17b66cbd3513c9a9f10074009a9d9fb5dae6985051 WatchSource:0}: Error finding container 8f5c822357e20c7a10f7ea17b66cbd3513c9a9f10074009a9d9fb5dae6985051: Status 404 returned error can't find the container with id 8f5c822357e20c7a10f7ea17b66cbd3513c9a9f10074009a9d9fb5dae6985051
Nov 02 21:45:01.280785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.280758    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:01.294793 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.294756    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:01.295756 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:01.295723    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f543c4_56c6_44bd_b4d6_298f2f329ea3.slice/crio-c81db29a9f9d445b054f2bd552be8b9d89b478df6e4ef800426b15ec439e5b23 WatchSource:0}: Error finding container c81db29a9f9d445b054f2bd552be8b9d89b478df6e4ef800426b15ec439e5b23: Status 404 returned error can't find the container with id c81db29a9f9d445b054f2bd552be8b9d89b478df6e4ef800426b15ec439e5b23
Nov 02 21:45:01.353012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.352946    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:01.355192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.355166    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:01.444695 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:01.444629    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7be4d7_1017_4f78_864b_875c96f3c69b.slice/crio-51ea5470346fb0d2b8805b9ddc9272b84085f1a0954514d448280772d9e80a17 WatchSource:0}: Error finding container 51ea5470346fb0d2b8805b9ddc9272b84085f1a0954514d448280772d9e80a17: Status 404 returned error can't find the container with id 51ea5470346fb0d2b8805b9ddc9272b84085f1a0954514d448280772d9e80a17
Nov 02 21:45:01.445055 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:45:01.445032    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899d3c52_70d8_4e38_8f4f_e992b4ed824e.slice/crio-3fecc232f7484645599ea3ff482f9d09eccefe048beffc7b336fb2e727afaf35 WatchSource:0}: Error finding container 3fecc232f7484645599ea3ff482f9d09eccefe048beffc7b336fb2e727afaf35: Status 404 returned error can't find the container with id 3fecc232f7484645599ea3ff482f9d09eccefe048beffc7b336fb2e727afaf35
Nov 02 21:45:01.745061 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.745023    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 02 21:45:01.745330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.745299    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 02 21:45:01.796522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.796420    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerStarted","Data":"569b31849b71f0fbc58dba55994a7567bb2066aa7959e12da8c255bc0fbb05a7"}
Nov 02 21:45:01.796522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.796505    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerStarted","Data":"6ba7b135f65cd5993a53de6d67791f4e5a3113d05b7b652e681f6f9e0ada033f"}
Nov 02 21:45:01.797727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.797697    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerStarted","Data":"c81db29a9f9d445b054f2bd552be8b9d89b478df6e4ef800426b15ec439e5b23"}
Nov 02 21:45:01.799285 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.799259    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerStarted","Data":"965979c66146de055d9cb40f1834a2120d259cef4e86a0e1ec79956c095fd74b"}
Nov 02 21:45:01.799398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.799299    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerStarted","Data":"8f5c822357e20c7a10f7ea17b66cbd3513c9a9f10074009a9d9fb5dae6985051"}
Nov 02 21:45:01.800605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.800581    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerStarted","Data":"51ea5470346fb0d2b8805b9ddc9272b84085f1a0954514d448280772d9e80a17"}
Nov 02 21:45:01.801633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.801613    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerStarted","Data":"3fecc232f7484645599ea3ff482f9d09eccefe048beffc7b336fb2e727afaf35"}
Nov 02 21:45:01.802815 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.802793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerStarted","Data":"9201ed40be81acbf85aaa39e1ceb07e6e690614816cd87278eab3e8484572405"}
Nov 02 21:45:01.802906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.802830    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerStarted","Data":"b9de02b662fd757c7b422265b4a4345bc63fd6a779e3e67f3ab0aa6b3c3933f3"}
Nov 02 21:45:01.928983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:01.928910    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" podStartSLOduration=1.9288862 podStartE2EDuration="1.9288862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:01.905338533 +0000 UTC m=+1931.449130449" watchObservedRunningTime="2026-11-02 21:45:01.9288862 +0000 UTC m=+1931.472678117"
Nov 02 21:45:02.029668 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.028718    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" podStartSLOduration=2.028694587 podStartE2EDuration="2.028694587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:02.025266504 +0000 UTC m=+1931.569058421" watchObservedRunningTime="2026-11-02 21:45:02.028694587 +0000 UTC m=+1931.572486506"
Nov 02 21:45:02.731386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.724415    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 02 21:45:02.731386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.724495    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 02 21:45:02.774133 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:02.774095    4888 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f543c4_56c6_44bd_b4d6_298f2f329ea3.slice/crio-8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f543c4_56c6_44bd_b4d6_298f2f329ea3.slice/crio-conmon-8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899d3c52_70d8_4e38_8f4f_e992b4ed824e.slice/crio-conmon-2c16bfc6ed9de601ee0b4a6b05e5507fcdefc2bb3dc7b360a9e7ef84c7bca185.scope\": RecentStats: unable to find data in memory cache]"
Nov 02 21:45:02.838156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.838106    4888 generic.go:356] "Generic (PLEG): container finished" podID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerID="569b31849b71f0fbc58dba55994a7567bb2066aa7959e12da8c255bc0fbb05a7" exitCode=60
Nov 02 21:45:02.838361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.838267    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerDied","Data":"569b31849b71f0fbc58dba55994a7567bb2066aa7959e12da8c255bc0fbb05a7"}
Nov 02 21:45:02.838841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.838782    4888 scope.go:122] "RemoveContainer" containerID="569b31849b71f0fbc58dba55994a7567bb2066aa7959e12da8c255bc0fbb05a7"
Nov 02 21:45:02.843409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.843243    4888 generic.go:356] "Generic (PLEG): container finished" podID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerID="8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5" exitCode=60
Nov 02 21:45:02.843505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.843404    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerDied","Data":"8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5"}
Nov 02 21:45:02.847200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.843601    4888 scope.go:122] "RemoveContainer" containerID="8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5"
Nov 02 21:45:02.847200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.846239    4888 generic.go:356] "Generic (PLEG): container finished" podID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerID="965979c66146de055d9cb40f1834a2120d259cef4e86a0e1ec79956c095fd74b" exitCode=60
Nov 02 21:45:02.847200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.846302    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerDied","Data":"965979c66146de055d9cb40f1834a2120d259cef4e86a0e1ec79956c095fd74b"}
Nov 02 21:45:02.847200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.846632    4888 scope.go:122] "RemoveContainer" containerID="965979c66146de055d9cb40f1834a2120d259cef4e86a0e1ec79956c095fd74b"
Nov 02 21:45:02.848617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.848593    4888 generic.go:356] "Generic (PLEG): container finished" podID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerID="98dcde46d3e2d16897d86a10c9e59e99bafb2d267a3611e30a216838514d783d" exitCode=60
Nov 02 21:45:02.848718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.848705    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerDied","Data":"98dcde46d3e2d16897d86a10c9e59e99bafb2d267a3611e30a216838514d783d"}
Nov 02 21:45:02.849044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.849029    4888 scope.go:122] "RemoveContainer" containerID="98dcde46d3e2d16897d86a10c9e59e99bafb2d267a3611e30a216838514d783d"
Nov 02 21:45:02.850996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.850971    4888 generic.go:356] "Generic (PLEG): container finished" podID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerID="2c16bfc6ed9de601ee0b4a6b05e5507fcdefc2bb3dc7b360a9e7ef84c7bca185" exitCode=60
Nov 02 21:45:02.851070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.851044    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerDied","Data":"2c16bfc6ed9de601ee0b4a6b05e5507fcdefc2bb3dc7b360a9e7ef84c7bca185"}
Nov 02 21:45:02.852302 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.852279    4888 scope.go:122] "RemoveContainer" containerID="2c16bfc6ed9de601ee0b4a6b05e5507fcdefc2bb3dc7b360a9e7ef84c7bca185"
Nov 02 21:45:02.856525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.856497    4888 generic.go:356] "Generic (PLEG): container finished" podID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerID="9201ed40be81acbf85aaa39e1ceb07e6e690614816cd87278eab3e8484572405" exitCode=60
Nov 02 21:45:02.856646 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.856534    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerDied","Data":"9201ed40be81acbf85aaa39e1ceb07e6e690614816cd87278eab3e8484572405"}
Nov 02 21:45:02.856931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.856912    4888 scope.go:122] "RemoveContainer" containerID="9201ed40be81acbf85aaa39e1ceb07e6e690614816cd87278eab3e8484572405"
Nov 02 21:45:02.952439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:02.952388    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" podStartSLOduration=2.952372752 podStartE2EDuration="2.952372752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:02.135136368 +0000 UTC m=+1931.678928285" watchObservedRunningTime="2026-11-02 21:45:02.952372752 +0000 UTC m=+1932.496164667"
Nov 02 21:45:03.736957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.723784    4888 patch_prober.go:29] interesting pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 02 21:45:03.736957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.723862    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 02 21:45:03.868294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.868252    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerStarted","Data":"e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"}
Nov 02 21:45:03.870596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.870562    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerStarted","Data":"974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"}
Nov 02 21:45:03.872425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.872396    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerStarted","Data":"3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"}
Nov 02 21:45:03.874289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.874267    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerStarted","Data":"1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"}
Nov 02 21:45:03.876034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.876015    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerStarted","Data":"94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"}
Nov 02 21:45:03.877722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.877702    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerStarted","Data":"8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"}
Nov 02 21:45:03.982842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:03.982776    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" podStartSLOduration=3.982758912 podStartE2EDuration="3.982758912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:03.981544421 +0000 UTC m=+1933.525336336" watchObservedRunningTime="2026-11-02 21:45:03.982758912 +0000 UTC m=+1933.526550829"
Nov 02 21:45:04.220117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.220044    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" podStartSLOduration=4.220023099 podStartE2EDuration="4.220023099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:04.215666674 +0000 UTC m=+1933.759458589" watchObservedRunningTime="2026-11-02 21:45:04.220023099 +0000 UTC m=+1933.763815017"
Nov 02 21:45:04.540559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.540452    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" podStartSLOduration=4.540436608 podStartE2EDuration="4.540436608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:04.537045837 +0000 UTC m=+1934.080837756" watchObservedRunningTime="2026-11-02 21:45:04.540436608 +0000 UTC m=+1934.084228523"
Nov 02 21:45:04.722122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.722092    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:45:04.722585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.722568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:45:04.723270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.723251    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:45:04.888045 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.887963    4888 generic.go:356] "Generic (PLEG): container finished" podID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerID="8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e" exitCode=60
Nov 02 21:45:04.888486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.888046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerDied","Data":"8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"}
Nov 02 21:45:04.888486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.888107    4888 scope.go:122] "RemoveContainer" containerID="569b31849b71f0fbc58dba55994a7567bb2066aa7959e12da8c255bc0fbb05a7"
Nov 02 21:45:04.888486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.888396    4888 scope.go:122] "RemoveContainer" containerID="8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"
Nov 02 21:45:04.888834 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.888813    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6_odh-ai-gateway-infra(34ad8fbe-d2de-485e-9fcc-0be5bb520c98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98"
Nov 02 21:45:04.890244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.890223    4888 generic.go:356] "Generic (PLEG): container finished" podID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerID="e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be" exitCode=60
Nov 02 21:45:04.890313 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.890296    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerDied","Data":"e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"}
Nov 02 21:45:04.890574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.890554    4888 scope.go:122] "RemoveContainer" containerID="e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"
Nov 02 21:45:04.890971 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.890942    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l_odh-ai-gateway-infra(e4f543c4-56c6-44bd-b4d6-298f2f329ea3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3"
Nov 02 21:45:04.892395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.892371    4888 generic.go:356] "Generic (PLEG): container finished" podID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerID="974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779" exitCode=60
Nov 02 21:45:04.892490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.892447    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerDied","Data":"974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"}
Nov 02 21:45:04.892710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.892692    4888 scope.go:122] "RemoveContainer" containerID="974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"
Nov 02 21:45:04.893059 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.893037    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c_odh-ai-gateway-infra(e7a25c28-24ba-4c89-88b8-d97935ba945d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d"
Nov 02 21:45:04.894347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.894327    4888 generic.go:356] "Generic (PLEG): container finished" podID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerID="3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924" exitCode=60
Nov 02 21:45:04.894426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.894406    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerDied","Data":"3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"}
Nov 02 21:45:04.894616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.894603    4888 scope.go:122] "RemoveContainer" containerID="3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"
Nov 02 21:45:04.894926 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.894902    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94_odh-ai-gateway-infra(9c7be4d7-1017-4f78-864b-875c96f3c69b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b"
Nov 02 21:45:04.896439 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.896416    4888 generic.go:356] "Generic (PLEG): container finished" podID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerID="1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042" exitCode=60
Nov 02 21:45:04.896532 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.896500    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerDied","Data":"1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"}
Nov 02 21:45:04.896748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.896709    4888 scope.go:122] "RemoveContainer" containerID="1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"
Nov 02 21:45:04.897022 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.896995    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2_odh-ai-gateway-infra(899d3c52-70d8-4e38-8f4f-e992b4ed824e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e"
Nov 02 21:45:04.898504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.898482    4888 generic.go:356] "Generic (PLEG): container finished" podID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerID="94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119" exitCode=60
Nov 02 21:45:04.898610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.898547    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerDied","Data":"94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"}
Nov 02 21:45:04.898752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.898736    4888 scope.go:122] "RemoveContainer" containerID="94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"
Nov 02 21:45:04.899024 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:04.898996    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj_odh-ai-gateway-infra(9a5688a3-9288-4a87-a7dd-40e031ea93ca)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca"
Nov 02 21:45:04.903055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.903037    4888 scope.go:122] "RemoveContainer" containerID="8eab721cfd3230cb60f4ad8cdf7a1b882262a71e52ee01dde381a2909e65e8b5"
Nov 02 21:45:04.917498 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.917480    4888 scope.go:122] "RemoveContainer" containerID="965979c66146de055d9cb40f1834a2120d259cef4e86a0e1ec79956c095fd74b"
Nov 02 21:45:04.934131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.934110    4888 scope.go:122] "RemoveContainer" containerID="98dcde46d3e2d16897d86a10c9e59e99bafb2d267a3611e30a216838514d783d"
Nov 02 21:45:04.948818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.948794    4888 scope.go:122] "RemoveContainer" containerID="2c16bfc6ed9de601ee0b4a6b05e5507fcdefc2bb3dc7b360a9e7ef84c7bca185"
Nov 02 21:45:04.979100 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:04.975565    4888 scope.go:122] "RemoveContainer" containerID="9201ed40be81acbf85aaa39e1ceb07e6e690614816cd87278eab3e8484572405"
Nov 02 21:45:05.908293 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.908266    4888 scope.go:122] "RemoveContainer" containerID="1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"
Nov 02 21:45:05.908752 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.908682    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2_odh-ai-gateway-infra(899d3c52-70d8-4e38-8f4f-e992b4ed824e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e"
Nov 02 21:45:05.909944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.909911    4888 scope.go:122] "RemoveContainer" containerID="94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"
Nov 02 21:45:05.910272 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.910249    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj_odh-ai-gateway-infra(9a5688a3-9288-4a87-a7dd-40e031ea93ca)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca"
Nov 02 21:45:05.911434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.911414    4888 scope.go:122] "RemoveContainer" containerID="8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"
Nov 02 21:45:05.911679 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.911651    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6_odh-ai-gateway-infra(34ad8fbe-d2de-485e-9fcc-0be5bb520c98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98"
Nov 02 21:45:05.913103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.913087    4888 scope.go:122] "RemoveContainer" containerID="e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"
Nov 02 21:45:05.913331 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.913308    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l_odh-ai-gateway-infra(e4f543c4-56c6-44bd-b4d6-298f2f329ea3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3"
Nov 02 21:45:05.914757 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.914736    4888 scope.go:122] "RemoveContainer" containerID="974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"
Nov 02 21:45:05.915013 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.914990    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c_odh-ai-gateway-infra(e7a25c28-24ba-4c89-88b8-d97935ba945d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d"
Nov 02 21:45:05.916310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:05.916291    4888 scope.go:122] "RemoveContainer" containerID="3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"
Nov 02 21:45:05.916563 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:05.916542    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94_odh-ai-gateway-infra(9c7be4d7-1017-4f78-864b-875c96f3c69b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b"
Nov 02 21:45:14.594123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:14.594083    4888 scope.go:122] "RemoveContainer" containerID="3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"
Nov 02 21:45:14.594533 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:14.594209    4888 scope.go:122] "RemoveContainer" containerID="1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"
Nov 02 21:45:15.600869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:15.599875    4888 scope.go:122] "RemoveContainer" containerID="e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"
Nov 02 21:45:16.001584 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.001551    4888 generic.go:356] "Generic (PLEG): container finished" podID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerID="9e0045cc5ee47af2c07aa22ae35ac20826dd3d2381dd842620fcae63dbb93297" exitCode=60
Nov 02 21:45:16.001760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.001623    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerDied","Data":"9e0045cc5ee47af2c07aa22ae35ac20826dd3d2381dd842620fcae63dbb93297"}
Nov 02 21:45:16.001760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.001689    4888 scope.go:122] "RemoveContainer" containerID="3c1703d4dfe5600a3aa1596dba85f986084cb6ed6f4b90d478e4d496267e7924"
Nov 02 21:45:16.002181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.002161    4888 scope.go:122] "RemoveContainer" containerID="9e0045cc5ee47af2c07aa22ae35ac20826dd3d2381dd842620fcae63dbb93297"
Nov 02 21:45:16.002573 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:16.002545    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94_odh-ai-gateway-infra(9c7be4d7-1017-4f78-864b-875c96f3c69b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b"
Nov 02 21:45:16.003797 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.003778    4888 generic.go:356] "Generic (PLEG): container finished" podID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerID="0b28daef9a50734634f256c3fc4e74a680613130951bbd0c93926f1df64ea578" exitCode=60
Nov 02 21:45:16.003922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.003903    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerDied","Data":"0b28daef9a50734634f256c3fc4e74a680613130951bbd0c93926f1df64ea578"}
Nov 02 21:45:16.004274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.004261    4888 scope.go:122] "RemoveContainer" containerID="0b28daef9a50734634f256c3fc4e74a680613130951bbd0c93926f1df64ea578"
Nov 02 21:45:16.004532 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:16.004514    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2_odh-ai-gateway-infra(899d3c52-70d8-4e38-8f4f-e992b4ed824e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e"
Nov 02 21:45:16.016713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.016684    4888 scope.go:122] "RemoveContainer" containerID="1863c46f616851e7e9530fc0d7659a9e43bb1ea81b8b40561edebf4ac299d042"
Nov 02 21:45:16.593330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:16.593294    4888 scope.go:122] "RemoveContainer" containerID="8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"
Nov 02 21:45:17.014022 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.013992    4888 generic.go:356] "Generic (PLEG): container finished" podID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerID="288dcd7a7ecd9e9457ab562ba3539e71806c662b8b85abe1979d6a6d9ab86aa8" exitCode=60
Nov 02 21:45:17.014473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.014066    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerDied","Data":"288dcd7a7ecd9e9457ab562ba3539e71806c662b8b85abe1979d6a6d9ab86aa8"}
Nov 02 21:45:17.014473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.014142    4888 scope.go:122] "RemoveContainer" containerID="e88a56f0910915bc115cf5a92a098ce955016dc4eaeaeed7ea03a2421d39c9be"
Nov 02 21:45:17.014621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.014575    4888 scope.go:122] "RemoveContainer" containerID="288dcd7a7ecd9e9457ab562ba3539e71806c662b8b85abe1979d6a6d9ab86aa8"
Nov 02 21:45:17.014962 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:17.014941    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l_odh-ai-gateway-infra(e4f543c4-56c6-44bd-b4d6-298f2f329ea3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3"
Nov 02 21:45:17.123563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.123533    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:17.231127 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.231076    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:17.325570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.325535    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:17.411393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.411354    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\" (UniqueName: \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\") pod \"9c7be4d7-1017-4f78-864b-875c96f3c69b\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") "
Nov 02 21:45:17.411579 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.411479    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\") pod \"9c7be4d7-1017-4f78-864b-875c96f3c69b\" (UID: \"9c7be4d7-1017-4f78-864b-875c96f3c69b\") "
Nov 02 21:45:17.412371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.412341    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca" pod "9c7be4d7-1017-4f78-864b-875c96f3c69b" (UID: "9c7be4d7-1017-4f78-864b-875c96f3c69b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:17.415085 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.415039    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5" pod "9c7be4d7-1017-4f78-864b-875c96f3c69b" (UID: "9c7be4d7-1017-4f78-864b-875c96f3c69b"). InnerVolumeSpecName "kube-api-access-szws5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:17.419321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.419298    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:17.513065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.512981    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\" (UniqueName: \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\") pod \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") "
Nov 02 21:45:17.513208 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.513085    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\") pod \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\" (UID: \"899d3c52-70d8-4e38-8f4f-e992b4ed824e\") "
Nov 02 21:45:17.513517 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.513491    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca" pod "899d3c52-70d8-4e38-8f4f-e992b4ed824e" (UID: "899d3c52-70d8-4e38-8f4f-e992b4ed824e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:17.513765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.513745    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c7be4d7-1017-4f78-864b-875c96f3c69b-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:17.513765 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.513765    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/899d3c52-70d8-4e38-8f4f-e992b4ed824e-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:17.513907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.513774    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szws5\" (UniqueName: \"kubernetes.io/projected/9c7be4d7-1017-4f78-864b-875c96f3c69b-kube-api-access-szws5\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:17.515266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.515245    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj" pod "899d3c52-70d8-4e38-8f4f-e992b4ed824e" (UID: "899d3c52-70d8-4e38-8f4f-e992b4ed824e"). InnerVolumeSpecName "kube-api-access-8z4fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:17.593332 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.593296    4888 scope.go:122] "RemoveContainer" containerID="974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"
Nov 02 21:45:17.615130 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:17.615092    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z4fj\" (UniqueName: \"kubernetes.io/projected/899d3c52-70d8-4e38-8f4f-e992b4ed824e-kube-api-access-8z4fj\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:18.028799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.028762    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerStarted","Data":"9a4ad520674d686d65a590b31123959703f8df99364232fadb8e1f350cba66d6"}
Nov 02 21:45:18.030195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.030175    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"
Nov 02 21:45:18.030300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.030225    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94" event={"ID":"9c7be4d7-1017-4f78-864b-875c96f3c69b","Type":"ContainerDied","Data":"51ea5470346fb0d2b8805b9ddc9272b84085f1a0954514d448280772d9e80a17"}
Nov 02 21:45:18.030300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.030270    4888 scope.go:122] "RemoveContainer" containerID="9e0045cc5ee47af2c07aa22ae35ac20826dd3d2381dd842620fcae63dbb93297"
Nov 02 21:45:18.031565 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.031545    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"
Nov 02 21:45:18.031659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.031570    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2" event={"ID":"899d3c52-70d8-4e38-8f4f-e992b4ed824e","Type":"ContainerDied","Data":"3fecc232f7484645599ea3ff482f9d09eccefe048beffc7b336fb2e727afaf35"}
Nov 02 21:45:18.034926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.034895    4888 generic.go:356] "Generic (PLEG): container finished" podID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerID="2834df5b10b41177eb9728ff810b64bcdddb75fda30aac85a5c5fca4d0bbff14" exitCode=60
Nov 02 21:45:18.035028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.034934    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerDied","Data":"2834df5b10b41177eb9728ff810b64bcdddb75fda30aac85a5c5fca4d0bbff14"}
Nov 02 21:45:18.035388 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.035351    4888 scope.go:122] "RemoveContainer" containerID="2834df5b10b41177eb9728ff810b64bcdddb75fda30aac85a5c5fca4d0bbff14"
Nov 02 21:45:18.035727 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:18.035702    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6_odh-ai-gateway-infra(34ad8fbe-d2de-485e-9fcc-0be5bb520c98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98"
Nov 02 21:45:18.042694 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.042592    4888 scope.go:122] "RemoveContainer" containerID="0b28daef9a50734634f256c3fc4e74a680613130951bbd0c93926f1df64ea578"
Nov 02 21:45:18.052741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.052720    4888 scope.go:122] "RemoveContainer" containerID="8b646c9c73f478692d3d6a7659bc1627db74deb27c30a07a39fc7964bbe4329e"
Nov 02 21:45:18.134400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.134298    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:18.249521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.249431    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:18.256693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.256653    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9a6cf2-29894265-ml7q2"]
Nov 02 21:45:18.277230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.277204    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:18.326057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.326023    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\" (UniqueName: \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\") pod \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") "
Nov 02 21:45:18.326252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.326137    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\") pod \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\" (UID: \"e4f543c4-56c6-44bd-b4d6-298f2f329ea3\") "
Nov 02 21:45:18.326527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.326505    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca" pod "e4f543c4-56c6-44bd-b4d6-298f2f329ea3" (UID: "e4f543c4-56c6-44bd-b4d6-298f2f329ea3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:18.326751 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.326735    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:18.328326 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.328300    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9" pod "e4f543c4-56c6-44bd-b4d6-298f2f329ea3" (UID: "e4f543c4-56c6-44bd-b4d6-298f2f329ea3"). InnerVolumeSpecName "kube-api-access-7nwt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:18.428486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.428453    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nwt9\" (UniqueName: \"kubernetes.io/projected/e4f543c4-56c6-44bd-b4d6-298f2f329ea3-kube-api-access-7nwt9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:18.463724 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.463690    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:18.471876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.471835    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dd6574-29894265-hvl94"]
Nov 02 21:45:18.593349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.593319    4888 scope.go:122] "RemoveContainer" containerID="94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"
Nov 02 21:45:18.603239 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.603208    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" path="/var/lib/kubelet/pods/899d3c52-70d8-4e38-8f4f-e992b4ed824e/volumes"
Nov 02 21:45:18.603711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:18.603697    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" path="/var/lib/kubelet/pods/9c7be4d7-1017-4f78-864b-875c96f3c69b/volumes"
Nov 02 21:45:19.044500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.044459    4888 generic.go:356] "Generic (PLEG): container finished" podID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerID="9a4ad520674d686d65a590b31123959703f8df99364232fadb8e1f350cba66d6" exitCode=60
Nov 02 21:45:19.045017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.044513    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerDied","Data":"9a4ad520674d686d65a590b31123959703f8df99364232fadb8e1f350cba66d6"}
Nov 02 21:45:19.045017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.044574    4888 scope.go:122] "RemoveContainer" containerID="974b91d64e8d2f1fe465260e040c00f2c431d558d6d436385ffa4d96d89be779"
Nov 02 21:45:19.045117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.045023    4888 scope.go:122] "RemoveContainer" containerID="9a4ad520674d686d65a590b31123959703f8df99364232fadb8e1f350cba66d6"
Nov 02 21:45:19.045393 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:19.045371    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c_odh-ai-gateway-infra(e7a25c28-24ba-4c89-88b8-d97935ba945d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d"
Nov 02 21:45:19.050158 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.050134    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"
Nov 02 21:45:19.050255 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.050155    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l" event={"ID":"e4f543c4-56c6-44bd-b4d6-298f2f329ea3","Type":"ContainerDied","Data":"c81db29a9f9d445b054f2bd552be8b9d89b478df6e4ef800426b15ec439e5b23"}
Nov 02 21:45:19.061215 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.061196    4888 scope.go:122] "RemoveContainer" containerID="288dcd7a7ecd9e9457ab562ba3539e71806c662b8b85abe1979d6a6d9ab86aa8"
Nov 02 21:45:19.151019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.150984    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:19.270621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.268809    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:19.273786 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.273752    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-c2dd3b-29894265-8z69l"]
Nov 02 21:45:19.367345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.367305    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:19.534440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.534366    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:19.646203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.646169    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\") pod \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") "
Nov 02 21:45:19.646480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.646455    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\" (UniqueName: \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\") pod \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\" (UID: \"34ad8fbe-d2de-485e-9fcc-0be5bb520c98\") "
Nov 02 21:45:19.646601 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.646586    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca" pod "34ad8fbe-d2de-485e-9fcc-0be5bb520c98" (UID: "34ad8fbe-d2de-485e-9fcc-0be5bb520c98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:19.661812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.661770    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96" pod "34ad8fbe-d2de-485e-9fcc-0be5bb520c98" (UID: "34ad8fbe-d2de-485e-9fcc-0be5bb520c98"). InnerVolumeSpecName "kube-api-access-wng96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:19.662316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.662295    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:19.662405 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:19.662322    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wng96\" (UniqueName: \"kubernetes.io/projected/34ad8fbe-d2de-485e-9fcc-0be5bb520c98-kube-api-access-wng96\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:20.114878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.109224    4888 generic.go:356] "Generic (PLEG): container finished" podID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerID="6e781657bc30f116e7a468791229b64a994e7e753ca98e26e17d647ecd9c21e7" exitCode=60
Nov 02 21:45:20.114878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.109423    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerDied","Data":"6e781657bc30f116e7a468791229b64a994e7e753ca98e26e17d647ecd9c21e7"}
Nov 02 21:45:20.114878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.109524    4888 scope.go:122] "RemoveContainer" containerID="94d1a15110c33697393b24b9e33c6d636c14e9f83980520a41327cfaa36a9119"
Nov 02 21:45:20.114878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.110159    4888 scope.go:122] "RemoveContainer" containerID="6e781657bc30f116e7a468791229b64a994e7e753ca98e26e17d647ecd9c21e7"
Nov 02 21:45:20.152946 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:20.110612    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj_odh-ai-gateway-infra(9a5688a3-9288-4a87-a7dd-40e031ea93ca)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca"
Nov 02 21:45:20.154576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.154493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6" event={"ID":"34ad8fbe-d2de-485e-9fcc-0be5bb520c98","Type":"ContainerDied","Data":"6ba7b135f65cd5993a53de6d67791f4e5a3113d05b7b652e681f6f9e0ada033f"}
Nov 02 21:45:20.154729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.154623    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"
Nov 02 21:45:20.189937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.189912    4888 scope.go:122] "RemoveContainer" containerID="2834df5b10b41177eb9728ff810b64bcdddb75fda30aac85a5c5fca4d0bbff14"
Nov 02 21:45:20.355652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.355624    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:20.429000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.428966    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:20.432911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.432883    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-ef6b43-29894265-8fgr6"]
Nov 02 21:45:20.446218 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.446192    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\") pod \"e7a25c28-24ba-4c89-88b8-d97935ba945d\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") "
Nov 02 21:45:20.446358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.446320    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\" (UniqueName: \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\") pod \"e7a25c28-24ba-4c89-88b8-d97935ba945d\" (UID: \"e7a25c28-24ba-4c89-88b8-d97935ba945d\") "
Nov 02 21:45:20.446575 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.446551    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca" pod "e7a25c28-24ba-4c89-88b8-d97935ba945d" (UID: "e7a25c28-24ba-4c89-88b8-d97935ba945d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:20.446807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.446793    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a25c28-24ba-4c89-88b8-d97935ba945d-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:20.448440 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.448417    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2" pod "e7a25c28-24ba-4c89-88b8-d97935ba945d" (UID: "e7a25c28-24ba-4c89-88b8-d97935ba945d"). InnerVolumeSpecName "kube-api-access-c26r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:20.548950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.548905    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c26r2\" (UniqueName: \"kubernetes.io/projected/e7a25c28-24ba-4c89-88b8-d97935ba945d-kube-api-access-c26r2\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:20.604309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.604233    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" path="/var/lib/kubelet/pods/34ad8fbe-d2de-485e-9fcc-0be5bb520c98/volumes"
Nov 02 21:45:20.604960 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:20.604938    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" path="/var/lib/kubelet/pods/e4f543c4-56c6-44bd-b4d6-298f2f329ea3/volumes"
Nov 02 21:45:21.192761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.192718    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"
Nov 02 21:45:21.193289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.192774    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c" event={"ID":"e7a25c28-24ba-4c89-88b8-d97935ba945d","Type":"ContainerDied","Data":"8f5c822357e20c7a10f7ea17b66cbd3513c9a9f10074009a9d9fb5dae6985051"}
Nov 02 21:45:21.193289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.192867    4888 scope.go:122] "RemoveContainer" containerID="9a4ad520674d686d65a590b31123959703f8df99364232fadb8e1f350cba66d6"
Nov 02 21:45:21.333647 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.333287    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:21.352453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.352415    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:21.378301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.378262    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-32de96-29894265-jc85c"]
Nov 02 21:45:21.511052 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.511023    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:21.564175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.564140    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\" (UniqueName: \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\") pod \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") "
Nov 02 21:45:21.564383 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.564199    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\") pod \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\" (UID: \"9a5688a3-9288-4a87-a7dd-40e031ea93ca\") "
Nov 02 21:45:21.565085 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.564619    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca" pod "9a5688a3-9288-4a87-a7dd-40e031ea93ca" (UID: "9a5688a3-9288-4a87-a7dd-40e031ea93ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:21.565085 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.565052    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5688a3-9288-4a87-a7dd-40e031ea93ca-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:21.566733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.566693    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w" pod "9a5688a3-9288-4a87-a7dd-40e031ea93ca" (UID: "9a5688a3-9288-4a87-a7dd-40e031ea93ca"). InnerVolumeSpecName "kube-api-access-lzr7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:21.666971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:21.666938    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzr7w\" (UniqueName: \"kubernetes.io/projected/9a5688a3-9288-4a87-a7dd-40e031ea93ca-kube-api-access-lzr7w\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:22.204997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.204965    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"
Nov 02 21:45:22.205478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.205006    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj" event={"ID":"9a5688a3-9288-4a87-a7dd-40e031ea93ca","Type":"ContainerDied","Data":"b9de02b662fd757c7b422265b4a4345bc63fd6a779e3e67f3ab0aa6b3c3933f3"}
Nov 02 21:45:22.205478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.205079    4888 scope.go:122] "RemoveContainer" containerID="6e781657bc30f116e7a468791229b64a994e7e753ca98e26e17d647ecd9c21e7"
Nov 02 21:45:22.265332 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:22.265295    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:45:22.363574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.363536    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:22.410603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.410567    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894265-nwvxj"]
Nov 02 21:45:22.603813 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.603726    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" path="/var/lib/kubelet/pods/9a5688a3-9288-4a87-a7dd-40e031ea93ca/volumes"
Nov 02 21:45:22.604382 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:22.604359    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" path="/var/lib/kubelet/pods/e7a25c28-24ba-4c89-88b8-d97935ba945d/volumes"
Nov 02 21:45:37.967691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:37.967634    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:45:38.010670 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:38.006762    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" containerID="cri-o://a68c6fb6ca3880430a70dc7fdac11f532da5172f5eea968f416877a3a5295893" gracePeriod=30
Nov 02 21:45:38.347209 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:38.347104    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:45:38.347688 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:38.347653    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" containerID="cri-o://5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065" gracePeriod=30
Nov 02 21:45:39.698880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:39.698014    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:39.800872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:39.800232    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:45:39.800872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:39.800313    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:45:39.842880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:39.841994    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.057063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.056949    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.081388 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.081338    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.268991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.268943    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.283050 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.283003    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.392187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.390831    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.403835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.403011    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.107:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.441414 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.423153    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.457494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.456918    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.107:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.611872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.611138    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.637225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.629040    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.703398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.703360    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.719336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.717837    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:45:40.790974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.790928    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.841583 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.840708    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:40.855871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.843119    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Nov 02 21:45:40.855871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:40.843476    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:45:41.276872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.273994    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:41.276872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.274530    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:45:41.405004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.404947    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:41.406026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.406000    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:45:41.421739 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.421674    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.107:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:41.422429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.422401    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:45:41.763874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.762987    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:45:41.763874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.763533    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:45:41.764470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.763889    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:41.764470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.764327    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:45:41.764470 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.764416    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:45:41.926880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.923524    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Nov 02 21:45:41.988210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:41.988143    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:42.165753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.000139    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:45:42.169458 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.169410    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": context deadline exceeded"
Nov 02 21:45:42.314976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.314931    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:42.323301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.323269    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 21:45:42.401998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.401953    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:42.405524 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.405472    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:42.428885 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.424194    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.107:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:42.486481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.486450    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:45:42.531728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.530573    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:45:42.742941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.709992    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 21:45:42.742941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.710054    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 21:45:42.742941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.710089    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:45:42.742941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.710119    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 21:45:42.742941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:42.710149    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 21:45:43.498054 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.498023    4888 generic.go:356] "Generic (PLEG): container finished" podID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerID="a68c6fb6ca3880430a70dc7fdac11f532da5172f5eea968f416877a3a5295893" exitCode=0
Nov 02 21:45:43.498818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.498792    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" event={"ID":"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3","Type":"ContainerDied","Data":"a68c6fb6ca3880430a70dc7fdac11f532da5172f5eea968f416877a3a5295893"}
Nov 02 21:45:43.669102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.646832    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:45:43.669102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.647151    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 21:45:43.695512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.695483    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:45:43.793779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.793739    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.793835    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794127    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:43.794478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794334    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794514 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794475    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794517    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794568    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794606    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794657 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794631    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794739    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\") pod \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\" (UID: \"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3\") "
Nov 02 21:45:43.794977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.794961    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:43.795558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.795535    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.795687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.795673    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.795779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.795640    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:43.795895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.795760    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:43.795997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.795809    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:43.797330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.797302    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:43.798575 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.798551    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "kube-api-access-h74jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:43.798709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.798692    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:45:43.798884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.798842    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy" pod "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" (UID: "b08e3782-a1f6-4ba7-b31c-d90a8560d7e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897148    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897196    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h74jp\" (UniqueName: \"kubernetes.io/projected/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-kube-api-access-h74jp\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897212    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897229    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897242    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897256    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:43.897469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:43.897271    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.453210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.449240    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:45:44.584857 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.584502    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.585673 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.585649    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.585841 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.585827    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.586004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.585988    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.586132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.586117    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.586258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.586245    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.586426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.586410    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.587147 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.586548    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.587438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.587418    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\") pod \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\" (UID: \"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f\") "
Nov 02 21:45:44.589380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.589355    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:44.590507 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.590256    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:44.596888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.595443    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:44.596888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.595763    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:44.597227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.597196    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:44.678704 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.676047    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708365    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708401    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708418    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708449    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708463    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.709082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.708477    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.734283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.709558    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:44.734283 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.709726    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "kube-api-access-cpwrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:44.800637 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.799947    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token" pod "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" (UID: "3fc80333-b572-4a7d-99b4-fdde6fc9ba7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:44.804673 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.804649    4888 generic.go:356] "Generic (PLEG): container finished" podID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerID="5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065" exitCode=0
Nov 02 21:45:44.804802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.804787    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" event={"ID":"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f","Type":"ContainerDied","Data":"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"}
Nov 02 21:45:44.804890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.804828    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp" event={"ID":"3fc80333-b572-4a7d-99b4-fdde6fc9ba7f","Type":"ContainerDied","Data":"ae166d6c9a6c5bf0edce3ce07a929c5cfe32763d57e2fec13f783500d57259c0"}
Nov 02 21:45:44.804890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.804867    4888 scope.go:122] "RemoveContainer" containerID="5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"
Nov 02 21:45:44.805013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.805001    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"
Nov 02 21:45:44.815995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.815716    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.815995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.815752    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.815995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.815769    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwrc\" (UniqueName: \"kubernetes.io/projected/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f-kube-api-access-cpwrc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:44.829969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.819338    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d" event={"ID":"b08e3782-a1f6-4ba7-b31c-d90a8560d7e3","Type":"ContainerDied","Data":"58b54b80b2ccb003ef648eb2bf50c4d57cec443fec74efeff259f342b9af1453"}
Nov 02 21:45:44.829969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.819463    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"
Nov 02 21:45:44.857160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.854095    4888 scope.go:122] "RemoveContainer" containerID="5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"
Nov 02 21:45:44.857160 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:44.854620    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065\": container with ID starting with 5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065 not found: ID does not exist" containerID="5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"
Nov 02 21:45:44.857160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.854658    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065"} err="failed to get container status \"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065\": rpc error: code = NotFound desc = could not find container \"5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065\": container with ID starting with 5eea5367b37e2f4c116f61227c5b6f41c86d94e22a0f6e3fe5d521457cb99065 not found: ID does not exist"
Nov 02 21:45:44.857160 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:44.854685    4888 scope.go:122] "RemoveContainer" containerID="a68c6fb6ca3880430a70dc7fdac11f532da5172f5eea968f416877a3a5295893"
Nov 02 21:45:45.005625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:45.005579    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:45:45.011152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:45.011111    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d"]
Nov 02 21:45:45.124603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:45.124491    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:45:45.124898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:45.124879    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp"]
Nov 02 21:45:46.604230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:46.604198    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" path="/var/lib/kubelet/pods/3fc80333-b572-4a7d-99b4-fdde6fc9ba7f/volumes"
Nov 02 21:45:46.605322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:46.605296    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" path="/var/lib/kubelet/pods/b08e3782-a1f6-4ba7-b31c-d90a8560d7e3/volumes"
Nov 02 21:45:48.433872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:48.428993    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:45:48.433872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:48.429064    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:45:50.879194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.879150    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:50.879930 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.879226    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:50.910752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.910713    4888 generic.go:356] "Generic (PLEG): container finished" podID="84744649-e77a-47bd-a63d-eddfb0961762" containerID="13efd7e8d80505038f3c838edeae5327c62705abcb8a8459b4653c114fc3541b" exitCode=0
Nov 02 21:45:50.910972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.910772    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerDied","Data":"13efd7e8d80505038f3c838edeae5327c62705abcb8a8459b4653c114fc3541b"}
Nov 02 21:45:50.911400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.911372    4888 scope.go:122] "RemoveContainer" containerID="13efd7e8d80505038f3c838edeae5327c62705abcb8a8459b4653c114fc3541b"
Nov 02 21:45:50.912970 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.912948    4888 generic.go:356] "Generic (PLEG): container finished" podID="150eec8f-a279-4e86-89ee-d9db889d304f" containerID="26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba" exitCode=0
Nov 02 21:45:50.913063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.912990    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerDied","Data":"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"}
Nov 02 21:45:50.913331 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.913316    4888 scope.go:122] "RemoveContainer" containerID="26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"
Nov 02 21:45:50.913420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.913312    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.913479 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.913464    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.913519 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.913500    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.915242 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.915206    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.915345 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.915292    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.915404 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.915340    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:50.956772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:50.956715    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:45:51.055157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:51.055075    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:45:51.926342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:51.925175    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerStarted","Data":"4df952acb5f09d33adaf2fe215a56733e71cdb8570c16ae8ab4e2e6b03c7dd46"}
Nov 02 21:45:51.966534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:51.963512    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerStarted","Data":"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"}
Nov 02 21:45:52.369313 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:52.369186    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:45:52.403607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.403568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:45:52.403821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.403736    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:52.403821 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.403784    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:52.721870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.709158    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:45:52.721870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.709306    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:52.721870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:52.709357    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:53.403928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.403538    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:53.403928 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.403606    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:53.704269 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.704170    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:53.704460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.704262    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:53.943709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.943668    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:45:53.944223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:53.944188    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" containerID="cri-o://ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b" gracePeriod=30
Nov 02 21:45:54.403694 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:54.403643    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:54.403952 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:54.403723    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:54.704963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:54.704880    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:54.704963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:54.704953    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:55.403525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.403478    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:55.403795 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.403545    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:55.557875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.551093    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:45:55.557875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.551157    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:45:55.704527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.704440    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:55.704527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.704502    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:55.958968 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:55.957918    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" podUID="dd98d189-ac09-4958-92bf-6e12289326d4" containerName="payload-processing" containerID="cri-o://0fbaa362490ac584b8f30f8957daa8bcf258da76e3e0d3e0a4001d6c2816bb35" gracePeriod=25
Nov 02 21:45:56.062569 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.060925    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" podUID="7352c232-4c95-45fe-b964-bc9ed4a58210" containerName="payload-pre-processing" containerID="cri-o://e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc" gracePeriod=25
Nov 02 21:45:56.145208 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.142162    4888 generic.go:356] "Generic (PLEG): container finished" podID="dd98d189-ac09-4958-92bf-6e12289326d4" containerID="0fbaa362490ac584b8f30f8957daa8bcf258da76e3e0d3e0a4001d6c2816bb35" exitCode=0
Nov 02 21:45:56.145208 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.142480    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" event={"ID":"dd98d189-ac09-4958-92bf-6e12289326d4","Type":"ContainerDied","Data":"0fbaa362490ac584b8f30f8957daa8bcf258da76e3e0d3e0a4001d6c2816bb35"}
Nov 02 21:45:56.403813 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.403772    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:56.404032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.403831    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:56.583733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.583427    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:45:56.637078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.628506    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\") pod \"dd98d189-ac09-4958-92bf-6e12289326d4\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") "
Nov 02 21:45:56.637078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.628621    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\" (UniqueName: \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\") pod \"dd98d189-ac09-4958-92bf-6e12289326d4\" (UID: \"dd98d189-ac09-4958-92bf-6e12289326d4\") "
Nov 02 21:45:56.637078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.630146    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume" pod "dd98d189-ac09-4958-92bf-6e12289326d4" (UID: "dd98d189-ac09-4958-92bf-6e12289326d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:56.682652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.682512    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc" pod "dd98d189-ac09-4958-92bf-6e12289326d4" (UID: "dd98d189-ac09-4958-92bf-6e12289326d4"). InnerVolumeSpecName "kube-api-access-5kqrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:56.705168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.705082    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:56.705168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.705131    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:56.750092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.734067    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd98d189-ac09-4958-92bf-6e12289326d4-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:56.750092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.734112    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kqrc\" (UniqueName: \"kubernetes.io/projected/dd98d189-ac09-4958-92bf-6e12289326d4-kube-api-access-5kqrc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:56.780840 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.780516    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:45:56.876750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.859892    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\" (UniqueName: \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\") pod \"7352c232-4c95-45fe-b964-bc9ed4a58210\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") "
Nov 02 21:45:56.876750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.859944    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\") pod \"7352c232-4c95-45fe-b964-bc9ed4a58210\" (UID: \"7352c232-4c95-45fe-b964-bc9ed4a58210\") "
Nov 02 21:45:56.876750 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.867320    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume" pod "7352c232-4c95-45fe-b964-bc9ed4a58210" (UID: "7352c232-4c95-45fe-b964-bc9ed4a58210"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:56.877571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.877432    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn" pod "7352c232-4c95-45fe-b964-bc9ed4a58210" (UID: "7352c232-4c95-45fe-b964-bc9ed4a58210"). InnerVolumeSpecName "kube-api-access-nkjzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:56.963873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.962377    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkjzn\" (UniqueName: \"kubernetes.io/projected/7352c232-4c95-45fe-b964-bc9ed4a58210-kube-api-access-nkjzn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:56.963873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:56.962421    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7352c232-4c95-45fe-b964-bc9ed4a58210-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:57.176504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.176021    4888 generic.go:356] "Generic (PLEG): container finished" podID="7352c232-4c95-45fe-b964-bc9ed4a58210" containerID="e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc" exitCode=0
Nov 02 21:45:57.176504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.176117    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"
Nov 02 21:45:57.176504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.176308    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" event={"ID":"7352c232-4c95-45fe-b964-bc9ed4a58210","Type":"ContainerDied","Data":"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"}
Nov 02 21:45:57.176504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.176351    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s" event={"ID":"7352c232-4c95-45fe-b964-bc9ed4a58210","Type":"ContainerDied","Data":"60480c6fed7d08439c25ac684a61402d12aa86eaf0a6820397385cf1bb43d9a0"}
Nov 02 21:45:57.176504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.176379    4888 scope.go:122] "RemoveContainer" containerID="e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"
Nov 02 21:45:57.180535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.180434    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh" event={"ID":"dd98d189-ac09-4958-92bf-6e12289326d4","Type":"ContainerDied","Data":"42a4e630623e22b56d8d72cc35bb98d55c591bcac2d39dba41ecee14e36ce20b"}
Nov 02 21:45:57.180965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.180753    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"
Nov 02 21:45:57.192287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.192245    4888 scope.go:122] "RemoveContainer" containerID="e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"
Nov 02 21:45:57.192639 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:57.192614    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc\": container with ID starting with e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc not found: ID does not exist" containerID="e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"
Nov 02 21:45:57.192722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.192647    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc"} err="failed to get container status \"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc\": rpc error: code = NotFound desc = could not find container \"e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc\": container with ID starting with e552ddffd85965f4f70fa18a5199c7f094c07a2cbd845412686f3e2879ddb5dc not found: ID does not exist"
Nov 02 21:45:57.192722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.192665    4888 scope.go:122] "RemoveContainer" containerID="0fbaa362490ac584b8f30f8957daa8bcf258da76e3e0d3e0a4001d6c2816bb35"
Nov 02 21:45:57.346084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.346045    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:45:57.389988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.389948    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh"]
Nov 02 21:45:57.405272 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.405241    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 21:45:57.405502 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.405481    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 21:45:57.467248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.465112    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:45:57.470600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.470562    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s"]
Nov 02 21:45:57.704926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.704886    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 02 21:45:57.705449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:57.704945    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 02 21:45:58.409663 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.409632    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:45:58.410027 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.410003    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:45:58.410943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.410924    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:45:58.602712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.602630    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7352c232-4c95-45fe-b964-bc9ed4a58210" path="/var/lib/kubelet/pods/7352c232-4c95-45fe-b964-bc9ed4a58210/volumes"
Nov 02 21:45:58.603101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.603086    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd98d189-ac09-4958-92bf-6e12289326d4" path="/var/lib/kubelet/pods/dd98d189-ac09-4958-92bf-6e12289326d4/volumes"
Nov 02 21:45:58.712523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.712483    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:45:58.713725 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.713478    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:45:58.714242 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:58.714221    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:45:59.174224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.174197    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:45:59.201777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.201741    4888 generic.go:356] "Generic (PLEG): container finished" podID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerID="ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b" exitCode=0
Nov 02 21:45:59.201999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.201800    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"
Nov 02 21:45:59.201999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.201822    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" event={"ID":"a42953e8-b649-4b08-8dda-c8439f0ac2a5","Type":"ContainerDied","Data":"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"}
Nov 02 21:45:59.201999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.201904    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x" event={"ID":"a42953e8-b649-4b08-8dda-c8439f0ac2a5","Type":"ContainerDied","Data":"b543a8f5c420e575818bd31f780552387c2cf5bdf4af8397728d64fcbc3eb536"}
Nov 02 21:45:59.201999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.201929    4888 scope.go:122] "RemoveContainer" containerID="ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"
Nov 02 21:45:59.204134 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.204110    4888 generic.go:356] "Generic (PLEG): container finished" podID="2448d982-164d-43fc-9ca1-01c160c65369" containerID="3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e" exitCode=0
Nov 02 21:45:59.204232 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.204174    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerDied","Data":"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"}
Nov 02 21:45:59.204690 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.204675    4888 scope.go:122] "RemoveContainer" containerID="3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"
Nov 02 21:45:59.206806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.206771    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:59.206917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.206888    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:59.206965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.206942    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:45:59.209172 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209150    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209190    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209234    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209266    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209451 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209286    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209451 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209433    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209606 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209579    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:59.209675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209597    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:59.209675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209639    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209677    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209703    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:59.209931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209865    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\") pod \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\" (UID: \"a42953e8-b649-4b08-8dda-c8439f0ac2a5\") "
Nov 02 21:45:59.209931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209910    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:59.210108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.209945    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:45:59.210338 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.210317    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.210409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.210339    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.210409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.210348    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.210409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.210357    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.210409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.210365    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.211732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.211711    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:45:59.211843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.211820    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "kube-api-access-ldvkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:59.212464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.212435    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:45:59.213155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.213129    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token" pod "a42953e8-b649-4b08-8dda-c8439f0ac2a5" (UID: "a42953e8-b649-4b08-8dda-c8439f0ac2a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:45:59.215449 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.215424    4888 scope.go:122] "RemoveContainer" containerID="ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"
Nov 02 21:45:59.215785 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:45:59.215758    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b\": container with ID starting with ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b not found: ID does not exist" containerID="ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"
Nov 02 21:45:59.215917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.215796    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b"} err="failed to get container status \"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b\": rpc error: code = NotFound desc = could not find container \"ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b\": container with ID starting with ef8cfe720705b080e50f53fc317006853156edb9311c67213ca17ef7f77c718b not found: ID does not exist"
Nov 02 21:45:59.312824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.312791    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.312824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.312825    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.312824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.312842    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldvkv\" (UniqueName: \"kubernetes.io/projected/a42953e8-b649-4b08-8dda-c8439f0ac2a5-kube-api-access-ldvkv\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.312824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.312870    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a42953e8-b649-4b08-8dda-c8439f0ac2a5-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:45:59.627265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.627225    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:45:59.630837 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:45:59.630802    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x"]
Nov 02 21:46:00.240385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:00.240337    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerStarted","Data":"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"}
Nov 02 21:46:00.603106 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:00.603009    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" path="/var/lib/kubelet/pods/a42953e8-b649-4b08-8dda-c8439f0ac2a5/volumes"
Nov 02 21:46:00.652689 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:00.652654    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:46:00.653880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:00.653834    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:00.653998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:00.653914    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:01.652484 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:01.652435    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:01.653055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:01.652494    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:02.652238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:02.652203    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:02.652425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:02.652263    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:03.033871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:03.032951    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:46:03.653152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:03.653117    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:03.653327 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:03.653172    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:04.652234 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:04.652200    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:04.652645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:04.652264    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:05.656321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:05.656277    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 21:46:05.656881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:05.656341    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 21:46:06.657494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:06.657456    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:46:06.658517 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:06.658494    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:46:07.476441 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:07.476395    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:46:07.476665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:07.476603    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" containerID="cri-o://976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6" gracePeriod=30
Nov 02 21:46:08.855609 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:08.855571    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:46:08.856067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:08.855644    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:46:12.613290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.613251    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:46:12.714583 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.714546    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:46:12.791703 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.791674    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:46:12.871638 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871596    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.871830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871666    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.871830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871807    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871899    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871934    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872017 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.871975    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872043    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872075    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:12.872183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872157    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872175    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:12.872183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872186    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\") pod \"347e20ea-7142-41bf-b0dd-30353da4c110\" (UID: \"347e20ea-7142-41bf-b0dd-30353da4c110\") "
Nov 02 21:46:12.872480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872102    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:12.872480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872380    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:12.872480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872391    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:12.872796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872772    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.872938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872799    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.872938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872812    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e20ea-7142-41bf-b0dd-30353da4c110-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.872938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872824    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.872938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.872837    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.874551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.874507    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:12.874658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.874546    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "kube-api-access-k7gx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:12.874658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.874603    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:12.874773 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.874662    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo" pod "347e20ea-7142-41bf-b0dd-30353da4c110" (UID: "347e20ea-7142-41bf-b0dd-30353da4c110"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:46:12.974608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.974524    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.974608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.974571    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7gx9\" (UniqueName: \"kubernetes.io/projected/347e20ea-7142-41bf-b0dd-30353da4c110-kube-api-access-k7gx9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.974608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.974589    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e20ea-7142-41bf-b0dd-30353da4c110-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:12.974608 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:12.974602    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e20ea-7142-41bf-b0dd-30353da4c110-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:13.398531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.398492    4888 generic.go:356] "Generic (PLEG): container finished" podID="347e20ea-7142-41bf-b0dd-30353da4c110" containerID="976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6" exitCode=0
Nov 02 21:46:13.398737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.398544    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"
Nov 02 21:46:13.398737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.398574    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" event={"ID":"347e20ea-7142-41bf-b0dd-30353da4c110","Type":"ContainerDied","Data":"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"}
Nov 02 21:46:13.398737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.398630    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp" event={"ID":"347e20ea-7142-41bf-b0dd-30353da4c110","Type":"ContainerDied","Data":"ace6f33d27ccb695573fabfc9560ace1a1745bf497b49d347bf1a66f1089905d"}
Nov 02 21:46:13.398737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.398651    4888 scope.go:122] "RemoveContainer" containerID="976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"
Nov 02 21:46:13.411015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.410993    4888 scope.go:122] "RemoveContainer" containerID="976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"
Nov 02 21:46:13.411414 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:13.411385    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6\": container with ID starting with 976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6 not found: ID does not exist" containerID="976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"
Nov 02 21:46:13.411486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.411425    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6"} err="failed to get container status \"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6\": rpc error: code = NotFound desc = could not find container \"976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6\": container with ID starting with 976e11534097cb8a988538365fbfede5d956a339bad0136a795cde0c369455a6 not found: ID does not exist"
Nov 02 21:46:13.556568 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.556531    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:46:13.561948 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:13.561912    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp"]
Nov 02 21:46:14.603890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:14.603828    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" path="/var/lib/kubelet/pods/347e20ea-7142-41bf-b0dd-30353da4c110/volumes"
Nov 02 21:46:17.614133 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.614094    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" podUID="599086a4-3738-4f77-9dae-f442b7062008" containerName="payload-processing" containerID="cri-o://6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650" gracePeriod=25
Nov 02 21:46:17.715766 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.715716    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" podUID="af948c30-2b3a-4bd3-96ca-7d240901b419" containerName="payload-pre-processing" containerID="cri-o://9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34" gracePeriod=25
Nov 02 21:46:17.781070 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.781036    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:46:17.903073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.898791    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:46:17.925777 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.925684    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:46:17.981898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.981824    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\") pod \"599086a4-3738-4f77-9dae-f442b7062008\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") "
Nov 02 21:46:17.982128 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.981962    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\" (UniqueName: \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\") pod \"599086a4-3738-4f77-9dae-f442b7062008\" (UID: \"599086a4-3738-4f77-9dae-f442b7062008\") "
Nov 02 21:46:17.983358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.983326    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume" pod "599086a4-3738-4f77-9dae-f442b7062008" (UID: "599086a4-3738-4f77-9dae-f442b7062008"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:17.986165 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:17.986137    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z" pod "599086a4-3738-4f77-9dae-f442b7062008" (UID: "599086a4-3738-4f77-9dae-f442b7062008"). InnerVolumeSpecName "kube-api-access-kc69z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:18.031723 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.031695    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:46:18.084915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.083862    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\" (UniqueName: \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\") pod \"af948c30-2b3a-4bd3-96ca-7d240901b419\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") "
Nov 02 21:46:18.084915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.084024    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\") pod \"af948c30-2b3a-4bd3-96ca-7d240901b419\" (UID: \"af948c30-2b3a-4bd3-96ca-7d240901b419\") "
Nov 02 21:46:18.084915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.084751    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume" pod "af948c30-2b3a-4bd3-96ca-7d240901b419" (UID: "af948c30-2b3a-4bd3-96ca-7d240901b419"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:18.084915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.084862    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/599086a4-3738-4f77-9dae-f442b7062008-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:18.084915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.084885    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc69z\" (UniqueName: \"kubernetes.io/projected/599086a4-3738-4f77-9dae-f442b7062008-kube-api-access-kc69z\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:18.086680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.086647    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q" pod "af948c30-2b3a-4bd3-96ca-7d240901b419" (UID: "af948c30-2b3a-4bd3-96ca-7d240901b419"). InnerVolumeSpecName "kube-api-access-tlm7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:18.187346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.187237    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlm7q\" (UniqueName: \"kubernetes.io/projected/af948c30-2b3a-4bd3-96ca-7d240901b419-kube-api-access-tlm7q\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:18.187346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.187282    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af948c30-2b3a-4bd3-96ca-7d240901b419-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:18.561984 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.561894    4888 generic.go:356] "Generic (PLEG): container finished" podID="599086a4-3738-4f77-9dae-f442b7062008" containerID="6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650" exitCode=0
Nov 02 21:46:18.562157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.562008    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"
Nov 02 21:46:18.562157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.562035    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" event={"ID":"599086a4-3738-4f77-9dae-f442b7062008","Type":"ContainerDied","Data":"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"}
Nov 02 21:46:18.562157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.562120    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb" event={"ID":"599086a4-3738-4f77-9dae-f442b7062008","Type":"ContainerDied","Data":"63c977a627ed00a6661f906c95fade071ddf25566a2ceec16b1fea5e68ad0550"}
Nov 02 21:46:18.562295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.562165    4888 scope.go:122] "RemoveContainer" containerID="6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"
Nov 02 21:46:18.563668 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.563641    4888 generic.go:356] "Generic (PLEG): container finished" podID="af948c30-2b3a-4bd3-96ca-7d240901b419" containerID="9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34" exitCode=0
Nov 02 21:46:18.563788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.563677    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"
Nov 02 21:46:18.563788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.563678    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" event={"ID":"af948c30-2b3a-4bd3-96ca-7d240901b419","Type":"ContainerDied","Data":"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"}
Nov 02 21:46:18.563899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.563805    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv" event={"ID":"af948c30-2b3a-4bd3-96ca-7d240901b419","Type":"ContainerDied","Data":"cca4653882f6cc47613cbf843a60c503a1d9ff518bc355775792bb4eb8300fd8"}
Nov 02 21:46:18.573585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.573565    4888 scope.go:122] "RemoveContainer" containerID="6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"
Nov 02 21:46:18.573931 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:18.573903    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650\": container with ID starting with 6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650 not found: ID does not exist" containerID="6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"
Nov 02 21:46:18.574037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.573940    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650"} err="failed to get container status \"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650\": rpc error: code = NotFound desc = could not find container \"6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650\": container with ID starting with 6adc0c51d552205a36041fe9a01988c1ae31371827c26761e4db1aa3b5121650 not found: ID does not exist"
Nov 02 21:46:18.574037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.573969    4888 scope.go:122] "RemoveContainer" containerID="9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"
Nov 02 21:46:18.584148 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.584007    4888 scope.go:122] "RemoveContainer" containerID="9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"
Nov 02 21:46:18.584426 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:18.584402    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34\": container with ID starting with 9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34 not found: ID does not exist" containerID="9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"
Nov 02 21:46:18.584500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.584435    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34"} err="failed to get container status \"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34\": rpc error: code = NotFound desc = could not find container \"9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34\": container with ID starting with 9617ae097528c34c1193a3b73d5edadf82139a740c94ba52fcd0a0d863f22b34 not found: ID does not exist"
Nov 02 21:46:18.719217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.719179    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:46:18.723159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.723127    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb"]
Nov 02 21:46:18.824950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.824814    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:46:18.829079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:18.829049    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv"]
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.570394    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571435    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571456    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571474    4888 cpu_manager.go:414] "containerMap: removing container" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571482    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571492    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571500    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571519    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571526    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571545    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571553    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571565    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571572    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571600    4888 cpu_manager.go:414] "containerMap: removing container" podUID="dd98d189-ac09-4958-92bf-6e12289326d4" containerName="payload-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571608    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd98d189-ac09-4958-92bf-6e12289326d4" containerName="payload-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571618    4888 cpu_manager.go:414] "containerMap: removing container" podUID="599086a4-3738-4f77-9dae-f442b7062008" containerName="payload-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571625    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="599086a4-3738-4f77-9dae-f442b7062008" containerName="payload-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571649    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571656    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571671    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571678    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571702    4888 cpu_manager.go:414] "containerMap: removing container" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571710    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571722    4888 cpu_manager.go:414] "containerMap: removing container" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571729    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571743    4888 cpu_manager.go:414] "containerMap: removing container" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571750    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571762    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571769    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571783    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571791    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571810    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571817    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571828    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571835    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571863    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7352c232-4c95-45fe-b964-bc9ed4a58210" containerName="payload-pre-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571871    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7352c232-4c95-45fe-b964-bc9ed4a58210" containerName="payload-pre-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571882    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571888    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571899    4888 cpu_manager.go:414] "containerMap: removing container" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571906    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571917    4888 cpu_manager.go:414] "containerMap: removing container" podUID="af948c30-2b3a-4bd3-96ca-7d240901b419" containerName="payload-pre-processing"
Nov 02 21:46:19.571871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.571935    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af948c30-2b3a-4bd3-96ca-7d240901b419" containerName="payload-pre-processing"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572132    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="af948c30-2b3a-4bd3-96ca-7d240901b419" containerName="payload-pre-processing"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572147    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572159    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572168    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572177    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572186    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572194    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572204    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572211    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572221    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd98d189-ac09-4958-92bf-6e12289326d4" containerName="payload-processing"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572232    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="599086a4-3738-4f77-9dae-f442b7062008" containerName="payload-processing"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572245    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572255    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572283    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572294    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572306    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572314    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b08e3782-a1f6-4ba7-b31c-d90a8560d7e3" containerName="istio-proxy"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572324    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572333    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572342    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a42953e8-b649-4b08-8dda-c8439f0ac2a5" containerName="istio-proxy"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572349    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7352c232-4c95-45fe-b964-bc9ed4a58210" containerName="payload-pre-processing"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572359    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fc80333-b572-4a7d-99b4-fdde6fc9ba7f" containerName="istio-proxy"
Nov 02 21:46:19.574229 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.572375    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="347e20ea-7142-41bf-b0dd-30353da4c110" containerName="istio-proxy"
Nov 02 21:46:19.695557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.695501    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:46:19.699549 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.699516    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701020    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701049    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4f543c4-56c6-44bd-b4d6-298f2f329ea3" containerName="cleanup"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701082    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701089    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701102    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.701119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701109    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701211    4888 cpu_manager.go:414] "containerMap: removing container" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701219    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="899d3c52-70d8-4e38-8f4f-e992b4ed824e" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701243    4888 cpu_manager.go:414] "containerMap: removing container" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701263    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34ad8fbe-d2de-485e-9fcc-0be5bb520c98" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701618    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c7be4d7-1017-4f78-864b-875c96f3c69b" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701642    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a5688a3-9288-4a87-a7dd-40e031ea93ca" containerName="cleanup"
Nov 02 21:46:19.701997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.701672    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7a25c28-24ba-4c89-88b8-d97935ba945d" containerName="cleanup"
Nov 02 21:46:19.710915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.709006    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:46:19.710915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.709289    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.722216 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.722157    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:46:19.767389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767345    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767392    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r4ph\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767421    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767561    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767615    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767642    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.767923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.767891    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.768103 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.768076    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.768233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.768117    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869230    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869275    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r4ph\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869334 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869305    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869337    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.869610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.869610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869433    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869616    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869656    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869677    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869735    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869772    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869797    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggzf\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869912    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869941    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.869976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.869972    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.870231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870008    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.870231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870095    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.870231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870226    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.870411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870250    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.870411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870279    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.870411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870345    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.870411 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.870599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.870484    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.872122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.872095    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.872544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.872525    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.925342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.925302    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.925607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.925583    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r4ph\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:19.972380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972338    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972381    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972453    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972480    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972531    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972569    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggzf\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.972659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972629    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972661    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972699    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972821    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973033 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.972918    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.973050    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.973218    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.973545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.973522    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.975065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.975039    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:19.975295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:19.975275    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:20.028794 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.028759    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:20.030419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.030388    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:20.031014 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.030992    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggzf\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\") pod \"e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") " pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:20.041176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.041145    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:20.457626 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:46:20.457593    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857bf1d9_a6b8_4d14_908e_8669c9647bef.slice/crio-f006b7b34452fca0c69db669840453a364512126814cc9b04d0a6f6afe21d71c WatchSource:0}: Error finding container f006b7b34452fca0c69db669840453a364512126814cc9b04d0a6f6afe21d71c: Status 404 returned error can't find the container with id f006b7b34452fca0c69db669840453a364512126814cc9b04d0a6f6afe21d71c
Nov 02 21:46:20.458597 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.458566    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:46:20.460791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.460753    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.460907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.460841    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.460907 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.460890    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.462400 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.462375    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:46:20.492818 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:46:20.492784    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573a1a40_d306_4239_b093_79fe496a41eb.slice/crio-d303eaef9e77b362ed9bd68b1673f0187d30da7f9476493a5c6a42577e115e79 WatchSource:0}: Error finding container d303eaef9e77b362ed9bd68b1673f0187d30da7f9476493a5c6a42577e115e79: Status 404 returned error can't find the container with id d303eaef9e77b362ed9bd68b1673f0187d30da7f9476493a5c6a42577e115e79
Nov 02 21:46:20.496129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.496077    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.496223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.496170    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.496223 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.496199    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:46:20.627565 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.627522    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="599086a4-3738-4f77-9dae-f442b7062008" path="/var/lib/kubelet/pods/599086a4-3738-4f77-9dae-f442b7062008/volumes"
Nov 02 21:46:20.628173 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.628150    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af948c30-2b3a-4bd3-96ca-7d240901b419" path="/var/lib/kubelet/pods/af948c30-2b3a-4bd3-96ca-7d240901b419/volumes"
Nov 02 21:46:20.682605 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.682552    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" event={"ID":"573a1a40-d306-4239-b093-79fe496a41eb","Type":"ContainerStarted","Data":"da758383770e9e2ecf2e7691935d71be6ad5cd51924568a453a4099ee630068a"}
Nov 02 21:46:20.682814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.682620    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" event={"ID":"573a1a40-d306-4239-b093-79fe496a41eb","Type":"ContainerStarted","Data":"d303eaef9e77b362ed9bd68b1673f0187d30da7f9476493a5c6a42577e115e79"}
Nov 02 21:46:20.685496 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.685461    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" event={"ID":"857bf1d9-a6b8-4d14-908e-8669c9647bef","Type":"ContainerStarted","Data":"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"}
Nov 02 21:46:20.685650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.685527    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" event={"ID":"857bf1d9-a6b8-4d14-908e-8669c9647bef","Type":"ContainerStarted","Data":"f006b7b34452fca0c69db669840453a364512126814cc9b04d0a6f6afe21d71c"}
Nov 02 21:46:20.798203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.798086    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podStartSLOduration=1.7980718470000001 podStartE2EDuration="1.798071847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:46:20.79495338 +0000 UTC m=+2010.338745300" watchObservedRunningTime="2026-11-02 21:46:20.798071847 +0000 UTC m=+2010.341863762"
Nov 02 21:46:20.911477 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:20.911401    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podStartSLOduration=1.911378255 podStartE2EDuration="1.911378255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:46:20.907171611 +0000 UTC m=+2010.450963527" watchObservedRunningTime="2026-11-02 21:46:20.911378255 +0000 UTC m=+2010.455170172"
Nov 02 21:46:21.029971 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.029934    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:21.031704 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.031673    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:21.031983 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.031954    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:21.043812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.043766    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:21.047869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.046506    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:21.047869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:21.046574    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:22.034883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.033977    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:22.034883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.034041    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:22.049393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.049353    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:22.049698 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.049669    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:22.287988 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:22.287903    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:46:22.782265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.782214    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerName="payload-processing" containerID="cri-o://c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2" gracePeriod=25
Nov 02 21:46:22.900123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:22.900056    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerName="payload-pre-processing" containerID="cri-o://0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4" gracePeriod=25
Nov 02 21:46:22.964080 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:46:22.964045    4888 logging.go:55] [core] [Channel #2003 SubChannel #2004]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.100:50052", ServerName: "10.132.0.100:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.100:50052: connect: connection refused"
Nov 02 21:46:22.965104 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:46:22.965064    4888 logging.go:55] [core] [Channel #2005 SubChannel #2006]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.99:50052", ServerName: "10.132.0.99:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.99:50052: connect: connection refused"
Nov 02 21:46:23.047351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.045526    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:23.047351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.045608    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:23.078378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.071972    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:23.078378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.072070    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:23.375941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.375738    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:46:23.386415 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.386377    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\" (UniqueName: \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\") pod \"82c2edc5-4fc0-4271-8de8-16c09325d313\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") "
Nov 02 21:46:23.386659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.386637    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\") pod \"82c2edc5-4fc0-4271-8de8-16c09325d313\" (UID: \"82c2edc5-4fc0-4271-8de8-16c09325d313\") "
Nov 02 21:46:23.387738 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.387709    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume" pod "82c2edc5-4fc0-4271-8de8-16c09325d313" (UID: "82c2edc5-4fc0-4271-8de8-16c09325d313"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:23.390309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.390281    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb" pod "82c2edc5-4fc0-4271-8de8-16c09325d313" (UID: "82c2edc5-4fc0-4271-8de8-16c09325d313"). InnerVolumeSpecName "kube-api-access-5jptb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:23.481077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.481048    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:46:23.488094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488062    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\" (UniqueName: \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\") pod \"5e2e2563-37ef-44af-b64d-6a50d34966cf\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") "
Nov 02 21:46:23.488245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488172    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\") pod \"5e2e2563-37ef-44af-b64d-6a50d34966cf\" (UID: \"5e2e2563-37ef-44af-b64d-6a50d34966cf\") "
Nov 02 21:46:23.488528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488505    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume" pod "5e2e2563-37ef-44af-b64d-6a50d34966cf" (UID: "5e2e2563-37ef-44af-b64d-6a50d34966cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:23.488803 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488784    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jptb\" (UniqueName: \"kubernetes.io/projected/82c2edc5-4fc0-4271-8de8-16c09325d313-kube-api-access-5jptb\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:23.488915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488804    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2e2563-37ef-44af-b64d-6a50d34966cf-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:23.488915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.488823    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c2edc5-4fc0-4271-8de8-16c09325d313-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:23.514880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.512226    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf" pod "5e2e2563-37ef-44af-b64d-6a50d34966cf" (UID: "5e2e2563-37ef-44af-b64d-6a50d34966cf"). InnerVolumeSpecName "kube-api-access-vvvxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:23.583991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.579946    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:46:23.583991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.580232    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" containerID="cri-o://a8d2615c232cfa2176bc832234ad30343cc9f0f62f7859409ed5b09ac769c5a7" gracePeriod=30
Nov 02 21:46:23.590171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.590133    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvvxf\" (UniqueName: \"kubernetes.io/projected/5e2e2563-37ef-44af-b64d-6a50d34966cf-kube-api-access-vvvxf\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:23.781860 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.781806    4888 generic.go:356] "Generic (PLEG): container finished" podID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerID="c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2" exitCode=0
Nov 02 21:46:23.782039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.781942    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"
Nov 02 21:46:23.782039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.781976    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" event={"ID":"82c2edc5-4fc0-4271-8de8-16c09325d313","Type":"ContainerDied","Data":"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"}
Nov 02 21:46:23.782039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.782028    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" event={"ID":"82c2edc5-4fc0-4271-8de8-16c09325d313","Type":"ContainerDied","Data":"02a84e982a7c4639e1c52930b08c21e8cd79112197b09b9268b5941daa1ff1e9"}
Nov 02 21:46:23.782227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.782059    4888 scope.go:122] "RemoveContainer" containerID="c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"
Nov 02 21:46:23.784044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.784005    4888 generic.go:356] "Generic (PLEG): container finished" podID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerID="0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4" exitCode=0
Nov 02 21:46:23.784270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.784247    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" event={"ID":"5e2e2563-37ef-44af-b64d-6a50d34966cf","Type":"ContainerDied","Data":"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"}
Nov 02 21:46:23.784347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.784307    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" event={"ID":"5e2e2563-37ef-44af-b64d-6a50d34966cf","Type":"ContainerDied","Data":"9845abb10bb8f9be8b669dc715eeba4f553cc8569eff34c201331d2f8df74ee1"}
Nov 02 21:46:23.784426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.784384    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"
Nov 02 21:46:23.796738 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.796714    4888 scope.go:122] "RemoveContainer" containerID="c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"
Nov 02 21:46:23.797144 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:23.797113    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2\": container with ID starting with c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2 not found: ID does not exist" containerID="c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"
Nov 02 21:46:23.797233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.797176    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2"} err="failed to get container status \"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2\": rpc error: code = NotFound desc = could not find container \"c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2\": container with ID starting with c57807c3487c3440ad828f3a9c453cd76f73e04017bf9a8c5fdecf04629a00b2 not found: ID does not exist"
Nov 02 21:46:23.797233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.797206    4888 scope.go:122] "RemoveContainer" containerID="0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"
Nov 02 21:46:23.867260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.867231    4888 scope.go:122] "RemoveContainer" containerID="0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"
Nov 02 21:46:23.867708 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:23.867674    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4\": container with ID starting with 0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4 not found: ID does not exist" containerID="0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"
Nov 02 21:46:23.867785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.867727    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4"} err="failed to get container status \"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4\": rpc error: code = NotFound desc = could not find container \"0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4\": container with ID starting with 0ad3f3006de772b79cc36c4378e35c12aabba89672565d4bc1bdb89f18d977a4 not found: ID does not exist"
Nov 02 21:46:23.939091 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.939034    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.99:50052\" within 1s: context deadline exceeded"
Nov 02 21:46:23.939817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.939766    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.100:50052\" within 1s: context deadline exceeded"
Nov 02 21:46:23.966224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.966155    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:46:23.973087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:23.973052    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd"]
Nov 02 21:46:24.051929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.033346    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:24.051929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.033409    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:24.051929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.046930    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:24.051929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.046984    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:24.070326 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.070270    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:46:24.074480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.074442    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db"]
Nov 02 21:46:24.601917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.601880    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" path="/var/lib/kubelet/pods/5e2e2563-37ef-44af-b64d-6a50d34966cf/volumes"
Nov 02 21:46:24.602468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:24.602450    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" path="/var/lib/kubelet/pods/82c2edc5-4fc0-4271-8de8-16c09325d313/volumes"
Nov 02 21:46:25.030881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.030323    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:25.030881 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.030389    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:25.044429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.044377    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:25.044614 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.044453    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:25.567616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.567545    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:46:25.567616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:25.567603    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:46:26.029733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:26.029266    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 02 21:46:26.029733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:26.029330    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 02 21:46:26.043135 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:26.042276    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 02 21:46:26.043135 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:26.042339    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 02 21:46:27.034192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.034160    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:27.034662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.034643    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:27.035492 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.035473    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:46:27.046312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.046287    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:27.046589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.046568    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:27.047268 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:27.047253    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:46:28.828643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:28.828601    4888 generic.go:356] "Generic (PLEG): container finished" podID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerID="a8d2615c232cfa2176bc832234ad30343cc9f0f62f7859409ed5b09ac769c5a7" exitCode=0
Nov 02 21:46:28.829100 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:28.828688    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" event={"ID":"7d5e927f-53ae-4334-b4a3-0b5935d362ae","Type":"ContainerDied","Data":"a8d2615c232cfa2176bc832234ad30343cc9f0f62f7859409ed5b09ac769c5a7"}
Nov 02 21:46:28.864483 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:28.864429    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:46:29.024438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024404    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024490    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024559    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024633 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024617    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024696    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024784    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024839    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:29.024955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024904    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.024955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024941    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.025115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.024994    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\") pod \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\" (UID: \"7d5e927f-53ae-4334-b4a3-0b5935d362ae\") "
Nov 02 21:46:29.025115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025014    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:29.025226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025210    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:29.025397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025369    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:29.025648 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025629    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.025743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025654    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.025743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025669    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.025743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.025683    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.027260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.027234    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:29.027546 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.027522    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:29.027656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.027544    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:46:29.028034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.028009    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:46:29.028280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.028256    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss" pod "7d5e927f-53ae-4334-b4a3-0b5935d362ae" (UID: "7d5e927f-53ae-4334-b4a3-0b5935d362ae"). InnerVolumeSpecName "kube-api-access-nz6ss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:29.127084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.127010    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.127084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.127037    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz6ss\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-kube-api-access-nz6ss\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.127084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.127049    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.127084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.127058    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.127084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.127067    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d5e927f-53ae-4334-b4a3-0b5935d362ae-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:29.839138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.839045    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"
Nov 02 21:46:29.839138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.839081    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns" event={"ID":"7d5e927f-53ae-4334-b4a3-0b5935d362ae","Type":"ContainerDied","Data":"9f9565d3d9c86af345de079f065009d32ba868f9984783ca528bb3911479a193"}
Nov 02 21:46:29.839653 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:29.839143    4888 scope.go:122] "RemoveContainer" containerID="a8d2615c232cfa2176bc832234ad30343cc9f0f62f7859409ed5b09ac769c5a7"
Nov 02 21:46:30.047946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:30.047907    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:46:30.094985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:30.094897    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns"]
Nov 02 21:46:30.602997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:30.602960    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" path="/var/lib/kubelet/pods/7d5e927f-53ae-4334-b4a3-0b5935d362ae/volumes"
Nov 02 21:46:52.260885 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:52.260828    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:46:53.116617 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:53.116582    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:46:53.225796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:53.225762    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:46:57.659675 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:57.659639    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:46:57.762623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:57.762585    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:46:58.117217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.117119    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" podUID="3f4f2d69-9649-47f3-beff-abc41b99330b" containerName="payload-processing" containerID="cri-o://f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b" gracePeriod=25
Nov 02 21:46:58.226392 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.226347    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" podUID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" containerName="payload-pre-processing" containerID="cri-o://064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048" gracePeriod=25
Nov 02 21:46:58.412000 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.411966    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:46:58.509943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.509912    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\" (UniqueName: \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\") pod \"3f4f2d69-9649-47f3-beff-abc41b99330b\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") "
Nov 02 21:46:58.510117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.510105    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\") pod \"3f4f2d69-9649-47f3-beff-abc41b99330b\" (UID: \"3f4f2d69-9649-47f3-beff-abc41b99330b\") "
Nov 02 21:46:58.510570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.510543    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume" pod "3f4f2d69-9649-47f3-beff-abc41b99330b" (UID: "3f4f2d69-9649-47f3-beff-abc41b99330b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:58.512077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.512053    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9" pod "3f4f2d69-9649-47f3-beff-abc41b99330b" (UID: "3f4f2d69-9649-47f3-beff-abc41b99330b"). InnerVolumeSpecName "kube-api-access-777k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:58.533434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.533403    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:46:58.611079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.611048    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\" (UniqueName: \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\") pod \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") "
Nov 02 21:46:58.611263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.611232    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\") pod \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\" (UID: \"2a23439f-3c9e-4c05-9a78-39b2f7c170c0\") "
Nov 02 21:46:58.611596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.611566    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume" pod "2a23439f-3c9e-4c05-9a78-39b2f7c170c0" (UID: "2a23439f-3c9e-4c05-9a78-39b2f7c170c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:46:58.611596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.611576    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-777k9\" (UniqueName: \"kubernetes.io/projected/3f4f2d69-9649-47f3-beff-abc41b99330b-kube-api-access-777k9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:58.611769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.611616    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4f2d69-9649-47f3-beff-abc41b99330b-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:58.613386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.613311    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f" pod "2a23439f-3c9e-4c05-9a78-39b2f7c170c0" (UID: "2a23439f-3c9e-4c05-9a78-39b2f7c170c0"). InnerVolumeSpecName "kube-api-access-2g79f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:46:58.712686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.712654    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:58.712686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:58.712683    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g79f\" (UniqueName: \"kubernetes.io/projected/2a23439f-3c9e-4c05-9a78-39b2f7c170c0-kube-api-access-2g79f\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:46:59.104709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.104623    4888 generic.go:356] "Generic (PLEG): container finished" podID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" containerID="064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048" exitCode=0
Nov 02 21:46:59.105194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.105165    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" event={"ID":"2a23439f-3c9e-4c05-9a78-39b2f7c170c0","Type":"ContainerDied","Data":"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"}
Nov 02 21:46:59.105370 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.105354    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf" event={"ID":"2a23439f-3c9e-4c05-9a78-39b2f7c170c0","Type":"ContainerDied","Data":"f01e8d04ec6ae9d262bd92f03b80763b81a8ede92d0be7e3708dc48e670d9060"}
Nov 02 21:46:59.105500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.105487    4888 scope.go:122] "RemoveContainer" containerID="064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"
Nov 02 21:46:59.105776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.105757    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"
Nov 02 21:46:59.153566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.136034    4888 generic.go:356] "Generic (PLEG): container finished" podID="3f4f2d69-9649-47f3-beff-abc41b99330b" containerID="f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b" exitCode=0
Nov 02 21:46:59.153566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.136116    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" event={"ID":"3f4f2d69-9649-47f3-beff-abc41b99330b","Type":"ContainerDied","Data":"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"}
Nov 02 21:46:59.153566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.136177    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn" event={"ID":"3f4f2d69-9649-47f3-beff-abc41b99330b","Type":"ContainerDied","Data":"c500fdbd5a9e64e3bf09680d7bf6d74421f0ff56ad4af885115d1c38b7e98805"}
Nov 02 21:46:59.153566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.136266    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"
Nov 02 21:46:59.224189 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.224160    4888 scope.go:122] "RemoveContainer" containerID="064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"
Nov 02 21:46:59.224983 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:59.224955    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048\": container with ID starting with 064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048 not found: ID does not exist" containerID="064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"
Nov 02 21:46:59.225245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.225215    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048"} err="failed to get container status \"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048\": rpc error: code = NotFound desc = could not find container \"064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048\": container with ID starting with 064215b7fe4000375f30216194c8ff0ea9dc497149ccecf3271ad73fbdf20048 not found: ID does not exist"
Nov 02 21:46:59.225398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.225386    4888 scope.go:122] "RemoveContainer" containerID="f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"
Nov 02 21:46:59.275705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.275678    4888 scope.go:122] "RemoveContainer" containerID="f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"
Nov 02 21:46:59.277750 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:46:59.277721    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b\": container with ID starting with f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b not found: ID does not exist" containerID="f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"
Nov 02 21:46:59.278007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.277977    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b"} err="failed to get container status \"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b\": rpc error: code = NotFound desc = could not find container \"f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b\": container with ID starting with f2eb26384e56d9792448d40a4449f8b623174d8e9cfe034fbb243d07aa2c063b not found: ID does not exist"
Nov 02 21:46:59.363570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.363479    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:46:59.388652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.388545    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn"]
Nov 02 21:46:59.503956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.503908    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:46:59.509079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:46:59.509042    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf"]
Nov 02 21:47:00.603442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:00.603404    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" path="/var/lib/kubelet/pods/2a23439f-3c9e-4c05-9a78-39b2f7c170c0/volumes"
Nov 02 21:47:00.603956 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:00.603939    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f4f2d69-9649-47f3-beff-abc41b99330b" path="/var/lib/kubelet/pods/3f4f2d69-9649-47f3-beff-abc41b99330b/volumes"
Nov 02 21:47:02.659940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:02.659896    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" podUID="7956a661-a777-4747-9716-80595f3b2634" containerName="payload-processing" containerID="cri-o://8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12" gracePeriod=25
Nov 02 21:47:02.763392 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:02.763321    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" podUID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" containerName="payload-pre-processing" containerID="cri-o://f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18" gracePeriod=25
Nov 02 21:47:02.955897 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:02.948953    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:47:03.041032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.041000    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:47:03.044558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.044532    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\") pod \"7956a661-a777-4747-9716-80595f3b2634\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") "
Nov 02 21:47:03.044700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.044578    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\" (UniqueName: \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\") pod \"7956a661-a777-4747-9716-80595f3b2634\" (UID: \"7956a661-a777-4747-9716-80595f3b2634\") "
Nov 02 21:47:03.044986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.044960    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume" pod "7956a661-a777-4747-9716-80595f3b2634" (UID: "7956a661-a777-4747-9716-80595f3b2634"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:47:03.045136 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.045117    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7956a661-a777-4747-9716-80595f3b2634-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:03.046711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.046689    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc" pod "7956a661-a777-4747-9716-80595f3b2634" (UID: "7956a661-a777-4747-9716-80595f3b2634"). InnerVolumeSpecName "kube-api-access-8rgdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:47:03.146726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.146688    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\") pod \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") "
Nov 02 21:47:03.146943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.146776    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\" (UniqueName: \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\") pod \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\" (UID: \"b6885c5c-256b-4432-b4a2-cfbe5618d2a7\") "
Nov 02 21:47:03.147112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.147085    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume" pod "b6885c5c-256b-4432-b4a2-cfbe5618d2a7" (UID: "b6885c5c-256b-4432-b4a2-cfbe5618d2a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:47:03.147158 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.147123    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rgdc\" (UniqueName: \"kubernetes.io/projected/7956a661-a777-4747-9716-80595f3b2634-kube-api-access-8rgdc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:03.148921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.148896    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m" pod "b6885c5c-256b-4432-b4a2-cfbe5618d2a7" (UID: "b6885c5c-256b-4432-b4a2-cfbe5618d2a7"). InnerVolumeSpecName "kube-api-access-7465m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:47:03.202564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.202468    4888 generic.go:356] "Generic (PLEG): container finished" podID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" containerID="f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18" exitCode=0
Nov 02 21:47:03.202564 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.202546    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"
Nov 02 21:47:03.202820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.202558    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" event={"ID":"b6885c5c-256b-4432-b4a2-cfbe5618d2a7","Type":"ContainerDied","Data":"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"}
Nov 02 21:47:03.202820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.202628    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r" event={"ID":"b6885c5c-256b-4432-b4a2-cfbe5618d2a7","Type":"ContainerDied","Data":"c6ca72cfa5cfc8c6d5294e6e2eacb3af55e23ca5952905ee90e4caff4d377909"}
Nov 02 21:47:03.202820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.202659    4888 scope.go:122] "RemoveContainer" containerID="f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"
Nov 02 21:47:03.204409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.204386    4888 generic.go:356] "Generic (PLEG): container finished" podID="7956a661-a777-4747-9716-80595f3b2634" containerID="8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12" exitCode=0
Nov 02 21:47:03.209346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.204510    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" event={"ID":"7956a661-a777-4747-9716-80595f3b2634","Type":"ContainerDied","Data":"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"}
Nov 02 21:47:03.209346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.204553    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t" event={"ID":"7956a661-a777-4747-9716-80595f3b2634","Type":"ContainerDied","Data":"c0cd327ec221050ed0f54562394c4d10253aeeea77535923c8f895bc8631a3bf"}
Nov 02 21:47:03.209346 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.204603    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"
Nov 02 21:47:03.216064 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.215712    4888 scope.go:122] "RemoveContainer" containerID="f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"
Nov 02 21:47:03.216177 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:47:03.216151    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18\": container with ID starting with f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18 not found: ID does not exist" containerID="f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"
Nov 02 21:47:03.216252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.216186    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18"} err="failed to get container status \"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18\": rpc error: code = NotFound desc = could not find container \"f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18\": container with ID starting with f2181f186acc24763d5972f5a7878b56c09150c9a0dc5d6a289d4bc59ad31c18 not found: ID does not exist"
Nov 02 21:47:03.216252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.216211    4888 scope.go:122] "RemoveContainer" containerID="8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"
Nov 02 21:47:03.227711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.227688    4888 scope.go:122] "RemoveContainer" containerID="8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"
Nov 02 21:47:03.228037 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:47:03.228014    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12\": container with ID starting with 8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12 not found: ID does not exist" containerID="8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"
Nov 02 21:47:03.228099 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.228046    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12"} err="failed to get container status \"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12\": rpc error: code = NotFound desc = could not find container \"8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12\": container with ID starting with 8a0166d2d3edef5f1f01e43bfc23db66becbe0354c89a0a63c80874a4f6f8f12 not found: ID does not exist"
Nov 02 21:47:03.247976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.247947    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:03.247976 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.247975    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7465m\" (UniqueName: \"kubernetes.io/projected/b6885c5c-256b-4432-b4a2-cfbe5618d2a7-kube-api-access-7465m\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:03.361051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.361014    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:47:03.364663 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.364638    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t"]
Nov 02 21:47:03.468116 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.468027    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:47:03.471314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:03.471289    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r"]
Nov 02 21:47:04.603417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:04.603374    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7956a661-a777-4747-9716-80595f3b2634" path="/var/lib/kubelet/pods/7956a661-a777-4747-9716-80595f3b2634/volumes"
Nov 02 21:47:04.604015 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:04.603998    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" path="/var/lib/kubelet/pods/b6885c5c-256b-4432-b4a2-cfbe5618d2a7/volumes"
Nov 02 21:47:21.375915 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:21.375875    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:47:21.478046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:21.478012    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:47:22.260185 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:47:22.260145    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:47:26.376935 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.376893    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" podUID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" containerName="payload-processing" containerID="cri-o://32fa0e7ff9e28740ac8dd693537fd7bb315b26820c14071f6e091e04022ac08b" gracePeriod=25
Nov 02 21:47:26.479210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.479146    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" podUID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" containerName="payload-pre-processing" containerID="cri-o://691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283" gracePeriod=25
Nov 02 21:47:26.497238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.497205    4888 generic.go:356] "Generic (PLEG): container finished" podID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" containerID="32fa0e7ff9e28740ac8dd693537fd7bb315b26820c14071f6e091e04022ac08b" exitCode=0
Nov 02 21:47:26.497368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.497283    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" event={"ID":"491b9c98-366b-4e89-b8c1-b2bb6444f3ba","Type":"ContainerDied","Data":"32fa0e7ff9e28740ac8dd693537fd7bb315b26820c14071f6e091e04022ac08b"}
Nov 02 21:47:26.673491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.673465    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:47:26.776472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.776211    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\") pod \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") "
Nov 02 21:47:26.776472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.776368    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\" (UniqueName: \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\") pod \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\" (UID: \"491b9c98-366b-4e89-b8c1-b2bb6444f3ba\") "
Nov 02 21:47:26.777073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.777045    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume" pod "491b9c98-366b-4e89-b8c1-b2bb6444f3ba" (UID: "491b9c98-366b-4e89-b8c1-b2bb6444f3ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:47:26.778802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.778770    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc" pod "491b9c98-366b-4e89-b8c1-b2bb6444f3ba" (UID: "491b9c98-366b-4e89-b8c1-b2bb6444f3ba"). InnerVolumeSpecName "kube-api-access-zgccc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:47:26.788351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.788325    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:47:26.877515 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.877482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\") pod \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") "
Nov 02 21:47:26.877693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.877674    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\" (UniqueName: \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\") pod \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\" (UID: \"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2\") "
Nov 02 21:47:26.877985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.877962    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume" pod "d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" (UID: "d4b4e7ea-dfa3-4e57-8edd-34935630a7c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:47:26.878192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.878176    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:26.878244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.878195    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:26.878244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.878204    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgccc\" (UniqueName: \"kubernetes.io/projected/491b9c98-366b-4e89-b8c1-b2bb6444f3ba-kube-api-access-zgccc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:26.879866 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.879828    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm" pod "d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" (UID: "d4b4e7ea-dfa3-4e57-8edd-34935630a7c2"). InnerVolumeSpecName "kube-api-access-ghznm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:47:26.979674 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:26.979641    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghznm\" (UniqueName: \"kubernetes.io/projected/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2-kube-api-access-ghznm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:47:27.506357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.506319    4888 generic.go:356] "Generic (PLEG): container finished" podID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" containerID="691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283" exitCode=0
Nov 02 21:47:27.506877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.506369    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"
Nov 02 21:47:27.506877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.506392    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" event={"ID":"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2","Type":"ContainerDied","Data":"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"}
Nov 02 21:47:27.506877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.506447    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms" event={"ID":"d4b4e7ea-dfa3-4e57-8edd-34935630a7c2","Type":"ContainerDied","Data":"9e7f6f2d21d9ab4818e7c67d1c500751a4b256aa0c3aef24dd04ca61627c7fe8"}
Nov 02 21:47:27.506877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.506468    4888 scope.go:122] "RemoveContainer" containerID="691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"
Nov 02 21:47:27.508180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.508156    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"
Nov 02 21:47:27.508291 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.508172    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt" event={"ID":"491b9c98-366b-4e89-b8c1-b2bb6444f3ba","Type":"ContainerDied","Data":"5e3a2555cfdbdaa8fb73de832012fd27e6ee2f21d8093cc7ca66790c381a4c88"}
Nov 02 21:47:27.517464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.517444    4888 scope.go:122] "RemoveContainer" containerID="691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"
Nov 02 21:47:27.517796 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:47:27.517773    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283\": container with ID starting with 691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283 not found: ID does not exist" containerID="691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"
Nov 02 21:47:27.517884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.517805    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283"} err="failed to get container status \"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283\": rpc error: code = NotFound desc = could not find container \"691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283\": container with ID starting with 691f0e5d1ecc03a7a521b63d9bec2a25b2f29409d5a252cdc5dc0f86d0275283 not found: ID does not exist"
Nov 02 21:47:27.517884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.517824    4888 scope.go:122] "RemoveContainer" containerID="32fa0e7ff9e28740ac8dd693537fd7bb315b26820c14071f6e091e04022ac08b"
Nov 02 21:47:27.665689 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.665652    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:47:27.675080 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.675042    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt"]
Nov 02 21:47:27.785063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.785034    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:47:27.788120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:27.788095    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms"]
Nov 02 21:47:28.601349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:28.601316    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" path="/var/lib/kubelet/pods/491b9c98-366b-4e89-b8c1-b2bb6444f3ba/volumes"
Nov 02 21:47:28.601926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:28.601808    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" path="/var/lib/kubelet/pods/d4b4e7ea-dfa3-4e57-8edd-34935630a7c2/volumes"
Nov 02 21:47:50.900099 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:50.900066    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:47:50.908408 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:47:50.908383    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:47:52.260617 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:47:52.260577    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:48:14.765497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:14.765419    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:48:14.766256 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:14.766211    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy" containerID="cri-o://3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72" gracePeriod=30
Nov 02 21:48:20.047558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.047480    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:48:20.088482 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088442    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088507    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088533    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088559    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088615    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088789    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088804    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088839    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088891    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088928    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.088944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.088933    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:20.089246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089084    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:20.089246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089090    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\") pod \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\" (UID: \"49f46504-204b-47b6-9fdc-e3f41b89cdbb\") "
Nov 02 21:48:20.089358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089334    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:48:20.089864 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089823    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.089940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089869    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.089940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089884    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.089940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089903    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.089940 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.089918    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.091300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.091270    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:20.091415 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.091297    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:48:20.091555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.091525    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "kube-api-access-fw5tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:20.091614 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.091565    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token" pod "49f46504-204b-47b6-9fdc-e3f41b89cdbb" (UID: "49f46504-204b-47b6-9fdc-e3f41b89cdbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:20.180753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.180719    4888 generic.go:356] "Generic (PLEG): container finished" podID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerID="3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72" exitCode=0
Nov 02 21:48:20.180972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.180789    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"
Nov 02 21:48:20.180972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.180817    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerDied","Data":"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"}
Nov 02 21:48:20.180972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.180916    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz" event={"ID":"49f46504-204b-47b6-9fdc-e3f41b89cdbb","Type":"ContainerDied","Data":"eec84a293de33f78966a38bac3660d0ae2fd2c125b0a17e8cb0c36bf7441572c"}
Nov 02 21:48:20.180972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.180960    4888 scope.go:122] "RemoveContainer" containerID="3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"
Nov 02 21:48:20.192501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.192459    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.192501 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.192500    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.192967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.192514    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw5tv\" (UniqueName: \"kubernetes.io/projected/49f46504-204b-47b6-9fdc-e3f41b89cdbb-kube-api-access-fw5tv\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.192967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.192528    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49f46504-204b-47b6-9fdc-e3f41b89cdbb-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:20.198112 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.198035    4888 scope.go:122] "RemoveContainer" containerID="2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"
Nov 02 21:48:20.209244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.209221    4888 scope.go:122] "RemoveContainer" containerID="3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"
Nov 02 21:48:20.209620 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:20.209587    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72\": container with ID starting with 3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72 not found: ID does not exist" containerID="3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"
Nov 02 21:48:20.209741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.209630    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72"} err="failed to get container status \"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72\": rpc error: code = NotFound desc = could not find container \"3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72\": container with ID starting with 3e1d655e0fcb639ab46045ed02890f05feddbbba7dc556ded410672d95f6cc72 not found: ID does not exist"
Nov 02 21:48:20.209741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.209656    4888 scope.go:122] "RemoveContainer" containerID="2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"
Nov 02 21:48:20.210017 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:20.209997    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429\": container with ID starting with 2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429 not found: ID does not exist" containerID="2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"
Nov 02 21:48:20.210098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.210027    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429"} err="failed to get container status \"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429\": rpc error: code = NotFound desc = could not find container \"2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429\": container with ID starting with 2f86665914abd8d2da37d5954245fcab6989f07dfbc2f0e9c99da74240823429 not found: ID does not exist"
Nov 02 21:48:20.338480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.338373    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:48:20.341812 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.341780    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz"]
Nov 02 21:48:20.601894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:20.601784    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" path="/var/lib/kubelet/pods/49f46504-204b-47b6-9fdc-e3f41b89cdbb/volumes"
Nov 02 21:48:22.262416 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:22.262376    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:48:23.469964 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:23.469920    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:48:23.470530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:23.470171    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" containerID="cri-o://d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1" gracePeriod=30
Nov 02 21:48:25.038113 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.038052    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:48:25.072880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.068177    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:48:25.279825 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.279787    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:48:25.280436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280419    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy"
Nov 02 21:48:25.280486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280438    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy"
Nov 02 21:48:25.280486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280454    4888 cpu_manager.go:414] "containerMap: removing container" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.280486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280460    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.280486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280476    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerName="payload-pre-processing"
Nov 02 21:48:25.280486 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280481    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280499    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3f4f2d69-9649-47f3-beff-abc41b99330b" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280505    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f4f2d69-9649-47f3-beff-abc41b99330b" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280512    4888 cpu_manager.go:414] "containerMap: removing container" podUID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280519    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280535    4888 cpu_manager.go:414] "containerMap: removing container" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280541    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280554    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280561    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280582    4888 cpu_manager.go:414] "containerMap: removing container" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280587    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280594    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280599    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280606    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7956a661-a777-4747-9716-80595f3b2634" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280610    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7956a661-a777-4747-9716-80595f3b2634" containerName="payload-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280621    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" containerName="payload-pre-processing"
Nov 02 21:48:25.280641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280628    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" containerName="payload-pre-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280759    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="82c2edc5-4fc0-4271-8de8-16c09325d313" containerName="payload-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280769    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4b4e7ea-dfa3-4e57-8edd-34935630a7c2" containerName="payload-pre-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280779    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7956a661-a777-4747-9716-80595f3b2634" containerName="payload-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280792    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a23439f-3c9e-4c05-9a78-39b2f7c170c0" containerName="payload-pre-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280800    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d5e927f-53ae-4334-b4a3-0b5935d362ae" containerName="istio-proxy"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280807    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280814    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e2e2563-37ef-44af-b64d-6a50d34966cf" containerName="payload-pre-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280833    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f4f2d69-9649-47f3-beff-abc41b99330b" containerName="payload-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280843    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="491b9c98-366b-4e89-b8c1-b2bb6444f3ba" containerName="payload-processing"
Nov 02 21:48:25.281181 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.280865    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6885c5c-256b-4432-b4a2-cfbe5618d2a7" containerName="payload-pre-processing"
Nov 02 21:48:25.285950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.285927    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.336095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.336058    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:48:25.337104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.337061    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7cc1b1\""
Nov 02 21:48:25.520798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.519561    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7cc1b1-dockercfg-ztxn9\""
Nov 02 21:48:25.523957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.523924    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9vcm\" (UniqueName: \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.524367 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.524327    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.558545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.558448    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:48:25.559831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.559804    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="49f46504-204b-47b6-9fdc-e3f41b89cdbb" containerName="istio-proxy"
Nov 02 21:48:25.568755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.564172    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.586187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.586157    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:48:25.626349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.626304    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txsmc\" (UniqueName: \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.626553 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.626360    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9vcm\" (UniqueName: \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.626802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.626771    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.626986 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.626961    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.627981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.627958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.681961 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.681931    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9vcm\" (UniqueName: \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\") pod \"payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.728353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.728315    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.728547 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.728503    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txsmc\" (UniqueName: \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.729084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.729065    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.782872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.782816    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txsmc\" (UniqueName: \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\") pod \"payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:25.899987 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.899902    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:25.904490 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:25.904136    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:26.284335 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:48:26.284292    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2772247b_ee45_43aa_bfbe_ec4697942d72.slice/crio-d994868d36d114106a20341be7d0a9bf1e1592c82b8c7bdc7de871123e00ae89 WatchSource:0}: Error finding container d994868d36d114106a20341be7d0a9bf1e1592c82b8c7bdc7de871123e00ae89: Status 404 returned error can't find the container with id d994868d36d114106a20341be7d0a9bf1e1592c82b8c7bdc7de871123e00ae89
Nov 02 21:48:26.284819 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:26.284779    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:48:26.328449 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:48:26.328414    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67265b28_0d37_439c_bf71_bf137fc4a470.slice/crio-164bc8f7e6a7db8eecb12b2409c2bccf05eedca6aecd1073ffbff2637fb46c75 WatchSource:0}: Error finding container 164bc8f7e6a7db8eecb12b2409c2bccf05eedca6aecd1073ffbff2637fb46c75: Status 404 returned error can't find the container with id 164bc8f7e6a7db8eecb12b2409c2bccf05eedca6aecd1073ffbff2637fb46c75
Nov 02 21:48:26.328733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:26.328713    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:48:27.229797 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.229755    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" event={"ID":"67265b28-0d37-439c-bf71-bf137fc4a470","Type":"ContainerStarted","Data":"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"}
Nov 02 21:48:27.230032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.229822    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" event={"ID":"67265b28-0d37-439c-bf71-bf137fc4a470","Type":"ContainerStarted","Data":"164bc8f7e6a7db8eecb12b2409c2bccf05eedca6aecd1073ffbff2637fb46c75"}
Nov 02 21:48:27.230032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.230006    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:27.231626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.231603    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" event={"ID":"2772247b-ee45-43aa-bfbe-ec4697942d72","Type":"ContainerStarted","Data":"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"}
Nov 02 21:48:27.231746 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.231640    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" event={"ID":"2772247b-ee45-43aa-bfbe-ec4697942d72","Type":"ContainerStarted","Data":"d994868d36d114106a20341be7d0a9bf1e1592c82b8c7bdc7de871123e00ae89"}
Nov 02 21:48:27.231746 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.231691    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:27.340521 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.340465    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" podStartSLOduration=2.340448241 podStartE2EDuration="2.340448241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:48:27.339420609 +0000 UTC m=+2136.883212528" watchObservedRunningTime="2026-11-02 21:48:27.340448241 +0000 UTC m=+2136.884240163"
Nov 02 21:48:27.480922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:27.480790    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" podStartSLOduration=2.480774226 podStartE2EDuration="2.480774226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:48:27.473269885 +0000 UTC m=+2137.017061802" watchObservedRunningTime="2026-11-02 21:48:27.480774226 +0000 UTC m=+2137.024566170"
Nov 02 21:48:28.784813 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.784787    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:48:28.836353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836321    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836374    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836411    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836567    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836595    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836676    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836805 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836745    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836874    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\") pod \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\" (UID: \"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c\") "
Nov 02 21:48:28.836988 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836939    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:28.837095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.836946    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:48:28.837095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837057    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:28.837310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837272    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:28.837419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837303    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:28.837588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837568    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.837679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837591    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.837679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837607    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.837679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837637    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.837679 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.837652    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.839355 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.839322    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:28.839491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.839404    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:48:28.839903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.839879    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:28.839903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.839890    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl" pod "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" (UID: "cd5f5cdb-9306-4828-9f7c-b3efe1455c7c"). InnerVolumeSpecName "kube-api-access-lhndl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:28.939519 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.939483    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.939519 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.939519    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhndl\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-kube-api-access-lhndl\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.939726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.939534    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:28.939726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:28.939543    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:29.261342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.261304    4888 generic.go:356] "Generic (PLEG): container finished" podID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerID="d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1" exitCode=0
Nov 02 21:48:29.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.261386    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"
Nov 02 21:48:29.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.261422    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" event={"ID":"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c","Type":"ContainerDied","Data":"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"}
Nov 02 21:48:29.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.261480    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6" event={"ID":"cd5f5cdb-9306-4828-9f7c-b3efe1455c7c","Type":"ContainerDied","Data":"546adf7b9600d22684b1436eac115157a5be38ac88ab9bce9019f79fdf60a1ea"}
Nov 02 21:48:29.261539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.261507    4888 scope.go:122] "RemoveContainer" containerID="d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"
Nov 02 21:48:29.274450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.274427    4888 scope.go:122] "RemoveContainer" containerID="d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"
Nov 02 21:48:29.274814 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:29.274781    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1\": container with ID starting with d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1 not found: ID does not exist" containerID="d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"
Nov 02 21:48:29.274997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.274826    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1"} err="failed to get container status \"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1\": rpc error: code = NotFound desc = could not find container \"d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1\": container with ID starting with d6d45641e55f511058c4dc0ef26a08e9553357e5f392681236362ae8d6409ac1 not found: ID does not exist"
Nov 02 21:48:29.424414 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.424373    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:48:29.469660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:29.469622    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6"]
Nov 02 21:48:30.601373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:30.601340    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" path="/var/lib/kubelet/pods/cd5f5cdb-9306-4828-9f7c-b3efe1455c7c/volumes"
Nov 02 21:48:38.255481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:38.255451    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:48:38.255900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:38.255519    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:48:40.376196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:40.376155    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:48:40.376667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:40.376377    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" podUID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" containerName="maas-api" containerID="cri-o://a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb" gracePeriod=30
Nov 02 21:48:40.880330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:40.880296    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:48:41.004373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.004280    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\") pod \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") "
Nov 02 21:48:41.004555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.004433    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") pod \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") "
Nov 02 21:48:41.004555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.004484    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\" (UniqueName: \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\") pod \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") "
Nov 02 21:48:41.004667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.004561    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\") pod \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\" (UID: \"3a22d739-3bf0-448b-9eb2-1e607ec5668c\") "
Nov 02 21:48:41.004720 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.004681    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca" pod "3a22d739-3bf0-448b-9eb2-1e607ec5668c" (UID: "3a22d739-3bf0-448b-9eb2-1e607ec5668c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:48:41.005151 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.005129    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a22d739-3bf0-448b-9eb2-1e607ec5668c-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:41.010896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.010842    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs" pod "3a22d739-3bf0-448b-9eb2-1e607ec5668c" (UID: "3a22d739-3bf0-448b-9eb2-1e607ec5668c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:48:41.011026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.010933    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj" pod "3a22d739-3bf0-448b-9eb2-1e607ec5668c" (UID: "3a22d739-3bf0-448b-9eb2-1e607ec5668c"). InnerVolumeSpecName "kube-api-access-rh2gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:41.011102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.011082    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls" pod "3a22d739-3bf0-448b-9eb2-1e607ec5668c" (UID: "3a22d739-3bf0-448b-9eb2-1e607ec5668c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:48:41.111943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.111050    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:41.111943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.111089    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh2gj\" (UniqueName: \"kubernetes.io/projected/3a22d739-3bf0-448b-9eb2-1e607ec5668c-kube-api-access-rh2gj\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:41.111943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.111105    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a22d739-3bf0-448b-9eb2-1e607ec5668c-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:41.362838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.362749    4888 generic.go:356] "Generic (PLEG): container finished" podID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" containerID="a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb" exitCode=0
Nov 02 21:48:41.362838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.362794    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"
Nov 02 21:48:41.362838 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.362811    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" event={"ID":"3a22d739-3bf0-448b-9eb2-1e607ec5668c","Type":"ContainerDied","Data":"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"}
Nov 02 21:48:41.363117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.362871    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8" event={"ID":"3a22d739-3bf0-448b-9eb2-1e607ec5668c","Type":"ContainerDied","Data":"4a62bbaddfe863c151b87b1b2e3e141eddd55ce9c5e2466c56fc02f151cb72ff"}
Nov 02 21:48:41.363117 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.362896    4888 scope.go:122] "RemoveContainer" containerID="a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"
Nov 02 21:48:41.376032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.376002    4888 scope.go:122] "RemoveContainer" containerID="a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"
Nov 02 21:48:41.378552 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:41.378346    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb\": container with ID starting with a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb not found: ID does not exist" containerID="a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"
Nov 02 21:48:41.378552 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.378393    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb"} err="failed to get container status \"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb\": rpc error: code = NotFound desc = could not find container \"a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb\": container with ID starting with a45e60dbf0bb7d37065f04aaa840c470bf438831e445842890c2d9bc11233efb not found: ID does not exist"
Nov 02 21:48:41.548992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.544935    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:48:41.550685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:41.550654    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d3b375-b765cd7dc-s94t8"]
Nov 02 21:48:42.118468 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:42.118417    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:48:42.119005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:42.118965    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy" containerID="cri-o://0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09" gracePeriod=30
Nov 02 21:48:42.603342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:42.603309    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" path="/var/lib/kubelet/pods/3a22d739-3bf0-448b-9eb2-1e607ec5668c/volumes"
Nov 02 21:48:47.405011 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.404985    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:48:47.435769 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.435729    4888 generic.go:356] "Generic (PLEG): container finished" podID="2448d982-164d-43fc-9ca1-01c160c65369" containerID="0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09" exitCode=0
Nov 02 21:48:47.435955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.435778    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerDied","Data":"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"}
Nov 02 21:48:47.435955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.435802    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"
Nov 02 21:48:47.435955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.435834    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7" event={"ID":"2448d982-164d-43fc-9ca1-01c160c65369","Type":"ContainerDied","Data":"b174fbbb618cda27a0d19c3513aa30349f19aa7135b3c55519c83ae83380cb56"}
Nov 02 21:48:47.435955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.435876    4888 scope.go:122] "RemoveContainer" containerID="0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"
Nov 02 21:48:47.447455 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.447412    4888 scope.go:122] "RemoveContainer" containerID="3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"
Nov 02 21:48:47.457621 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.457603    4888 scope.go:122] "RemoveContainer" containerID="0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"
Nov 02 21:48:47.457925 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:47.457903    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09\": container with ID starting with 0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09 not found: ID does not exist" containerID="0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"
Nov 02 21:48:47.457997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.457934    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09"} err="failed to get container status \"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09\": rpc error: code = NotFound desc = could not find container \"0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09\": container with ID starting with 0d60f2fa9c76786dd68b3bddd8821a38715ca10e70c284291b5680fa71d9fb09 not found: ID does not exist"
Nov 02 21:48:47.457997 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.457953    4888 scope.go:122] "RemoveContainer" containerID="3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"
Nov 02 21:48:47.458219 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:47.458201    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e\": container with ID starting with 3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e not found: ID does not exist" containerID="3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"
Nov 02 21:48:47.458264 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.458225    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e"} err="failed to get container status \"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e\": rpc error: code = NotFound desc = could not find container \"3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e\": container with ID starting with 3201d968881f1715dc7983242034b70e4500507a307fe12b9115ac6b38fb860e not found: ID does not exist"
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.532579    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.532784    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.532864    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.532906    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.532969    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.533103    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.533152    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.533219    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.538491 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.533289    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\") pod \"2448d982-164d-43fc-9ca1-01c160c65369\" (UID: \"2448d982-164d-43fc-9ca1-01c160c65369\") "
Nov 02 21:48:47.552876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.544090    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:48:47.552876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.544272    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:47.552876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.544410    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:47.552876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.544584    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:47.552876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.544777    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:47.555055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.554993    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:47.579114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.565910    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:48:47.583882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.582128    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "kube-api-access-bdlvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:47.589867 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.584401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token" pod "2448d982-164d-43fc-9ca1-01c160c65369" (UID: "2448d982-164d-43fc-9ca1-01c160c65369"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651063    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2448d982-164d-43fc-9ca1-01c160c65369-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651103    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651121    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdlvz\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-kube-api-access-bdlvz\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651136    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2448d982-164d-43fc-9ca1-01c160c65369-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651150    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651164    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651178    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651190    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2448d982-164d-43fc-9ca1-01c160c65369-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.661429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.651203    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2448d982-164d-43fc-9ca1-01c160c65369-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:47.873798 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.873690    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:48:47.877782 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:47.877751    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7"]
Nov 02 21:48:48.635023 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:48.634929    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2448d982-164d-43fc-9ca1-01c160c65369" path="/var/lib/kubelet/pods/2448d982-164d-43fc-9ca1-01c160c65369/volumes"
Nov 02 21:48:49.236998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:49.236965    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:48:49.237230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:49.237184    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy" containerID="cri-o://b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18" gracePeriod=30
Nov 02 21:48:52.260515 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:52.260476    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:48:54.510764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.510701    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:48:54.533356 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.533327    4888 generic.go:356] "Generic (PLEG): container finished" podID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerID="b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18" exitCode=0
Nov 02 21:48:54.533511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.533378    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"
Nov 02 21:48:54.533511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.533400    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerDied","Data":"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"}
Nov 02 21:48:54.533511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.533458    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76" event={"ID":"e977df1a-c89d-45e5-b0e1-499fa3ad2f54","Type":"ContainerDied","Data":"45262bb68410adb96d941510c5fc5c4ce5d99a6353dea7ad07bc1a62b5833bff"}
Nov 02 21:48:54.533511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.533487    4888 scope.go:122] "RemoveContainer" containerID="b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"
Nov 02 21:48:54.543391 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.543371    4888 scope.go:122] "RemoveContainer" containerID="aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"
Nov 02 21:48:54.552065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.552049    4888 scope.go:122] "RemoveContainer" containerID="b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"
Nov 02 21:48:54.552322 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:54.552301    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18\": container with ID starting with b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18 not found: ID does not exist" containerID="b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"
Nov 02 21:48:54.552407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.552335    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18"} err="failed to get container status \"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18\": rpc error: code = NotFound desc = could not find container \"b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18\": container with ID starting with b63c857adee61ca046bd6f30a13046d2821e76131e47f7db5b9046e17c542f18 not found: ID does not exist"
Nov 02 21:48:54.552407 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.552359    4888 scope.go:122] "RemoveContainer" containerID="aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"
Nov 02 21:48:54.552638 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:48:54.552619    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd\": container with ID starting with aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd not found: ID does not exist" containerID="aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"
Nov 02 21:48:54.552683 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.552645    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd"} err="failed to get container status \"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd\": rpc error: code = NotFound desc = could not find container \"aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd\": container with ID starting with aea9a200215599cd0117648de019041d2e1e8baa0b681948bee75cfb90abf9cd not found: ID does not exist"
Nov 02 21:48:54.676139 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676108    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676171    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676195    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676246    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676320    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676359    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676467    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676516    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:54.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676542    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676565    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:54.676785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676571    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:54.676785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676583    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\") pod \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\" (UID: \"e977df1a-c89d-45e5-b0e1-499fa3ad2f54\") "
Nov 02 21:48:54.676785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676658    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:54.677021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.676806    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:48:54.677248 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.677224    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.677333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.677252    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.677333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.677265    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.677333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.677276    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.677333 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.677288    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.678656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.678634    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:54.679166 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.679140    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "kube-api-access-jwzn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:48:54.679266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.679169    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:48:54.679266 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.679242    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo" pod "e977df1a-c89d-45e5-b0e1-499fa3ad2f54" (UID: "e977df1a-c89d-45e5-b0e1-499fa3ad2f54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:48:54.778914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.778796    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.778914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.778829    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwzn9\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-kube-api-access-jwzn9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.778914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.778843    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.778914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.778877    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977df1a-c89d-45e5-b0e1-499fa3ad2f54-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:48:54.953075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.953035    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:48:54.957505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:54.957471    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76"]
Nov 02 21:48:56.603417 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:48:56.603381    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" path="/var/lib/kubelet/pods/e977df1a-c89d-45e5-b0e1-499fa3ad2f54/volumes"
Nov 02 21:49:00.652909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:00.652864    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:49:05.653252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:05.653203    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" podUID="c8f67e1a-38b8-4e78-8208-8110b7160b71" containerName="payload-processing" containerID="cri-o://3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340" gracePeriod=25
Nov 02 21:49:05.935806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:05.935780    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:49:06.026923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.026844    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\") pod \"c8f67e1a-38b8-4e78-8208-8110b7160b71\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") "
Nov 02 21:49:06.027122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.027106    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\" (UniqueName: \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\") pod \"c8f67e1a-38b8-4e78-8208-8110b7160b71\" (UID: \"c8f67e1a-38b8-4e78-8208-8110b7160b71\") "
Nov 02 21:49:06.027222 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.027197    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume" pod "c8f67e1a-38b8-4e78-8208-8110b7160b71" (UID: "c8f67e1a-38b8-4e78-8208-8110b7160b71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:49:06.027556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.027539    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8f67e1a-38b8-4e78-8208-8110b7160b71-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:06.029280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.029259    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb" pod "c8f67e1a-38b8-4e78-8208-8110b7160b71" (UID: "c8f67e1a-38b8-4e78-8208-8110b7160b71"). InnerVolumeSpecName "kube-api-access-hczjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:06.129245 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.129208    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hczjb\" (UniqueName: \"kubernetes.io/projected/c8f67e1a-38b8-4e78-8208-8110b7160b71-kube-api-access-hczjb\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:06.676176 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.676137    4888 generic.go:356] "Generic (PLEG): container finished" podID="c8f67e1a-38b8-4e78-8208-8110b7160b71" containerID="3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340" exitCode=0
Nov 02 21:49:06.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.676189    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"
Nov 02 21:49:06.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.676215    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" event={"ID":"c8f67e1a-38b8-4e78-8208-8110b7160b71","Type":"ContainerDied","Data":"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"}
Nov 02 21:49:06.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.676284    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk" event={"ID":"c8f67e1a-38b8-4e78-8208-8110b7160b71","Type":"ContainerDied","Data":"a2654e2d1b5f9a4371311c19b064901111f1d6849aef82a0e2c6e6fd9eb4ce0e"}
Nov 02 21:49:06.676557 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.676312    4888 scope.go:122] "RemoveContainer" containerID="3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"
Nov 02 21:49:06.686506 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.686485    4888 scope.go:122] "RemoveContainer" containerID="3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"
Nov 02 21:49:06.686839 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:49:06.686820    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340\": container with ID starting with 3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340 not found: ID does not exist" containerID="3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"
Nov 02 21:49:06.686943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.686870    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340"} err="failed to get container status \"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340\": rpc error: code = NotFound desc = could not find container \"3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340\": container with ID starting with 3174e545b7f92d35d07dfbe4aa793d776f30e260cfb93b7e808e06aad1123340 not found: ID does not exist"
Nov 02 21:49:06.908460 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.908389    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:49:06.919807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:06.919767    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk"]
Nov 02 21:49:08.602972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:08.602879    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8f67e1a-38b8-4e78-8208-8110b7160b71" path="/var/lib/kubelet/pods/c8f67e1a-38b8-4e78-8208-8110b7160b71/volumes"
Nov 02 21:49:22.260631 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:49:22.260586    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:49:27.351378 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351343    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:49:27.351942 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351929    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" containerName="maas-api"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351944    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" containerName="maas-api"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351969    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351974    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351983    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.351987    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352008    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352013    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352020    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c8f67e1a-38b8-4e78-8208-8110b7160b71" containerName="payload-processing"
Nov 02 21:49:27.352024 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352025    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8f67e1a-38b8-4e78-8208-8110b7160b71" containerName="payload-processing"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352047    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352051    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352140    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd5f5cdb-9306-4828-9f7c-b3efe1455c7c" containerName="istio-proxy"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352152    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8f67e1a-38b8-4e78-8208-8110b7160b71" containerName="payload-processing"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352161    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352174    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352180    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a22d739-3bf0-448b-9eb2-1e607ec5668c" containerName="maas-api"
Nov 02 21:49:27.352348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.352188    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:27.355742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.355722    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.523779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.523746    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fltmf\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.523995 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.523799    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.523992    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524046 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524032    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524054    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524214    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524338    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524404    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.524476 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.524435    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.540965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.540934    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:49:27.625318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625233    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625318 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625282    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625399    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625449    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fltmf\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625500    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625623    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625642    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625727 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625678    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625728    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.625947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.625791    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.626051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.626018    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.626258 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.626238    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.626335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.626256    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.626372 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.626347    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.627831 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.627805    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.628095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.628079    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.679793 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.679754    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.679994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.679832    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fltmf\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:27.969963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:27.969921    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:28.392027 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.392000    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:49:28.392356 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:49:28.392305    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2beee3_1fb9_4254_a409_357a3bfe0daf.slice/crio-2e2cbf90accd0e00a1740aab918d6a4fa0d081b8d6033a4f77bcbb27432c2e8e WatchSource:0}: Error finding container 2e2cbf90accd0e00a1740aab918d6a4fa0d081b8d6033a4f77bcbb27432c2e8e: Status 404 returned error can't find the container with id 2e2cbf90accd0e00a1740aab918d6a4fa0d081b8d6033a4f77bcbb27432c2e8e
Nov 02 21:49:28.394975 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.394944    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:28.395044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.395010    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:28.395044 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.395038    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:28.865071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.865029    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" event={"ID":"ac2beee3-1fb9-4254-a409-357a3bfe0daf","Type":"ContainerStarted","Data":"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"}
Nov 02 21:49:28.865287 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.865094    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" event={"ID":"ac2beee3-1fb9-4254-a409-357a3bfe0daf","Type":"ContainerStarted","Data":"2e2cbf90accd0e00a1740aab918d6a4fa0d081b8d6033a4f77bcbb27432c2e8e"}
Nov 02 21:49:28.971039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:28.970998    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:29.071153 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:29.071099    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podStartSLOduration=2.071081974 podStartE2EDuration="2.071081974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:49:29.022780759 +0000 UTC m=+2198.566572675" watchObservedRunningTime="2026-11-02 21:49:29.071081974 +0000 UTC m=+2198.614873890"
Nov 02 21:49:29.975785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:29.975588    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:29.976366 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:29.976336    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:30.970403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:30.970365    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:30.970593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:30.970441    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:31.970519 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:31.970483    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:31.970977 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:31.970549    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:32.867611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.867570    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:49:32.868220 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.868165    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy" containerID="cri-o://4df952acb5f09d33adaf2fe215a56733e71cdb8570c16ae8ab4e2e6b03c7dd46" gracePeriod=30
Nov 02 21:49:32.933874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.911605    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:49:32.934748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.934718    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:49:32.936115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.936089    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy" containerID="cri-o://da758383770e9e2ecf2e7691935d71be6ad5cd51924568a453a4099ee630068a" gracePeriod=30
Nov 02 21:49:32.947670 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.935089    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy" containerID="cri-o://4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde" gracePeriod=30
Nov 02 21:49:32.976311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.976271    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:32.976957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:32.976922    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:33.970467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:33.970431    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:33.970658 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:33.970502    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:34.153709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.153672    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:49:34.154634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.154602    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:34.154634 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.154633    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2448d982-164d-43fc-9ca1-01c160c65369" containerName="istio-proxy"
Nov 02 21:49:34.154835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.154820    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e977df1a-c89d-45e5-b0e1-499fa3ad2f54" containerName="istio-proxy"
Nov 02 21:49:34.158616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.158594    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195109    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195200    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195260    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195317    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195349    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195408    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195457    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195539    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmnt\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.195879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.195612    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.232137 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.232039    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:49:34.265191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.265131    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d0946d-openshift-default-dockercfg-cp45b\""
Nov 02 21:49:34.321875 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.298745    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.322288 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.322244    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.394060 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.394005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.394361 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.394333    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.394576 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.394558    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.394758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.394725    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.394994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.394979    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmnt\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.398957 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.398912    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.406261 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.406186    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.430872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.430821    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.457868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.431187    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.457868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.322880    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.457868 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.432189    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.458335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.458252    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.526443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.523795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.526443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.524043    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.594413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.594363    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.620065 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.602190    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmnt\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\") pod \"e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") " pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.775385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.775337    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:34.970597 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.970558    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 02 21:49:34.970806 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:34.970616    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 02 21:49:35.273630 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:49:35.273595    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63d53d7_46ef_4203_895b_2c7d4db0945b.slice/crio-b6a3720023c7a9907b9127a23d2a0d406a36a4d0292ee0ad1403660b1c817b7a WatchSource:0}: Error finding container b6a3720023c7a9907b9127a23d2a0d406a36a4d0292ee0ad1403660b1c817b7a: Status 404 returned error can't find the container with id b6a3720023c7a9907b9127a23d2a0d406a36a4d0292ee0ad1403660b1c817b7a
Nov 02 21:49:35.276061 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.276031    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:49:35.276551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.276518    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:35.276642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.276598    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:35.276705 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.276643    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:35.976121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.976088    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:35.976571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.976550    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:35.977356 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:35.977339    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:49:36.067691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:36.067600    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerStarted","Data":"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"}
Nov 02 21:49:36.067979 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:36.067699    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerStarted","Data":"b6a3720023c7a9907b9127a23d2a0d406a36a4d0292ee0ad1403660b1c817b7a"}
Nov 02 21:49:36.249041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:36.248926    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podStartSLOduration=2.248909685 podStartE2EDuration="2.248909685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:49:36.245798884 +0000 UTC m=+2205.789590802" watchObservedRunningTime="2026-11-02 21:49:36.248909685 +0000 UTC m=+2205.792701601"
Nov 02 21:49:36.775781 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:36.775738    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:36.782109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:36.782081    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:37.073929 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:37.073827    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:37.075072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:37.075046    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:49:38.097898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.097590    4888 generic.go:356] "Generic (PLEG): container finished" podID="573a1a40-d306-4239-b093-79fe496a41eb" containerID="da758383770e9e2ecf2e7691935d71be6ad5cd51924568a453a4099ee630068a" exitCode=0
Nov 02 21:49:38.097898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.097843    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" event={"ID":"573a1a40-d306-4239-b093-79fe496a41eb","Type":"ContainerDied","Data":"da758383770e9e2ecf2e7691935d71be6ad5cd51924568a453a4099ee630068a"}
Nov 02 21:49:38.100626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.100594    4888 generic.go:356] "Generic (PLEG): container finished" podID="84744649-e77a-47bd-a63d-eddfb0961762" containerID="4df952acb5f09d33adaf2fe215a56733e71cdb8570c16ae8ab4e2e6b03c7dd46" exitCode=0
Nov 02 21:49:38.100782 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.100664    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerDied","Data":"4df952acb5f09d33adaf2fe215a56733e71cdb8570c16ae8ab4e2e6b03c7dd46"}
Nov 02 21:49:38.100782 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.100721    4888 scope.go:122] "RemoveContainer" containerID="13efd7e8d80505038f3c838edeae5327c62705abcb8a8459b4653c114fc3541b"
Nov 02 21:49:38.208996 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.208098    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:49:38.227123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.227098    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:49:38.303899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.303841    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:49:38.309019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.308994    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309068    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.309138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309107    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.309224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309204    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.309260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309248    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.309306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309286    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309391    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309422 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309414    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309440    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309473    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309514    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309519    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309583    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.309669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309629    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309707    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309723    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309735    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309747    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309795    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309835    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309896    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309943    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\") pod \"84744649-e77a-47bd-a63d-eddfb0961762\" (UID: \"84744649-e77a-47bd-a63d-eddfb0961762\") "
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309945    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309968    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.309979    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\") pod \"573a1a40-d306-4239-b093-79fe496a41eb\" (UID: \"573a1a40-d306-4239-b093-79fe496a41eb\") "
Nov 02 21:49:38.310774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310260    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.310774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310750    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310776    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310792    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310807    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310821    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310835    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84744649-e77a-47bd-a63d-eddfb0961762-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310881    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.310912 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.310895    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.311741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.311712    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.311879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.311784    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:49:38.312200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312176    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.312290 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312260    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:49:38.312566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312542    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.312566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312556    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:49:38.312702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312594    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "kube-api-access-fggzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.312702 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312641    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.312963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.312947    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy" pod "573a1a40-d306-4239-b093-79fe496a41eb" (UID: "573a1a40-d306-4239-b093-79fe496a41eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.313570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.313542    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4" pod "84744649-e77a-47bd-a63d-eddfb0961762" (UID: "84744649-e77a-47bd-a63d-eddfb0961762"). InnerVolumeSpecName "kube-api-access-nzrq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.412300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412262    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412315    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412391    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412470    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412523    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412565    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412606    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.412759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412697    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.412759 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412730    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.413007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.412764    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\") pod \"857bf1d9-a6b8-4d14-908e-8669c9647bef\" (UID: \"857bf1d9-a6b8-4d14-908e-8669c9647bef\") "
Nov 02 21:49:38.413098 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.413056    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:49:38.413316 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.413283    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.413612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.413570    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.414242 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414210    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414636    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84744649-e77a-47bd-a63d-eddfb0961762-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414661    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414676    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84744649-e77a-47bd-a63d-eddfb0961762-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414691    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414705    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/573a1a40-d306-4239-b093-79fe496a41eb-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414721    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzrq4\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-kube-api-access-nzrq4\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414738    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/573a1a40-d306-4239-b093-79fe496a41eb-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414752    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/573a1a40-d306-4239-b093-79fe496a41eb-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414766    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414782    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414795    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84744649-e77a-47bd-a63d-eddfb0961762-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414806    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414818    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857bf1d9-a6b8-4d14-908e-8669c9647bef-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414831    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggzf\" (UniqueName: \"kubernetes.io/projected/573a1a40-d306-4239-b093-79fe496a41eb-kube-api-access-fggzf\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.414902 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.414879    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.415729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.415320    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "kube-api-access-8r4ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.415729 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.415539    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:49:38.416431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.416410    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:49:38.416743 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.416728    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo" pod "857bf1d9-a6b8-4d14-908e-8669c9647bef" (UID: "857bf1d9-a6b8-4d14-908e-8669c9647bef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:49:38.516551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.516457    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.516551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.516500    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.516551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.516519    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r4ph\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-kube-api-access-8r4ph\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:38.516551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:38.516534    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857bf1d9-a6b8-4d14-908e-8669c9647bef-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:49:39.108534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.108496    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl" event={"ID":"573a1a40-d306-4239-b093-79fe496a41eb","Type":"ContainerDied","Data":"d303eaef9e77b362ed9bd68b1673f0187d30da7f9476493a5c6a42577e115e79"}
Nov 02 21:49:39.109078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.108510    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"
Nov 02 21:49:39.109078 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.108596    4888 scope.go:122] "RemoveContainer" containerID="da758383770e9e2ecf2e7691935d71be6ad5cd51924568a453a4099ee630068a"
Nov 02 21:49:39.110013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.109992    4888 generic.go:356] "Generic (PLEG): container finished" podID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerID="4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde" exitCode=0
Nov 02 21:49:39.110102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.110057    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"
Nov 02 21:49:39.110102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.110077    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" event={"ID":"857bf1d9-a6b8-4d14-908e-8669c9647bef","Type":"ContainerDied","Data":"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"}
Nov 02 21:49:39.110230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.110140    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k" event={"ID":"857bf1d9-a6b8-4d14-908e-8669c9647bef","Type":"ContainerDied","Data":"f006b7b34452fca0c69db669840453a364512126814cc9b04d0a6f6afe21d71c"}
Nov 02 21:49:39.111889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.111867    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"
Nov 02 21:49:39.112005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.111905    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc" event={"ID":"84744649-e77a-47bd-a63d-eddfb0961762","Type":"ContainerDied","Data":"11095b7cb98ccb44aeb664a456b81fcd2ed33b82739fa91c01515429e1b3d667"}
Nov 02 21:49:39.119706 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.119685    4888 scope.go:122] "RemoveContainer" containerID="4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"
Nov 02 21:49:39.129943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.129918    4888 scope.go:122] "RemoveContainer" containerID="4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"
Nov 02 21:49:39.130250 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:49:39.130222    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde\": container with ID starting with 4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde not found: ID does not exist" containerID="4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"
Nov 02 21:49:39.130362 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.130261    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde"} err="failed to get container status \"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde\": rpc error: code = NotFound desc = could not find container \"4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde\": container with ID starting with 4c254b5d0067a93f8937d7a81b5a7a515bcd57bffcbdbb19f92513d1515c1dde not found: ID does not exist"
Nov 02 21:49:39.130362 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.130280    4888 scope.go:122] "RemoveContainer" containerID="4df952acb5f09d33adaf2fe215a56733e71cdb8570c16ae8ab4e2e6b03c7dd46"
Nov 02 21:49:39.267418 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.267336    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:49:39.314392 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.314358    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc"]
Nov 02 21:49:39.377204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.377159    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:49:39.380277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.380247    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k"]
Nov 02 21:49:39.483395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.483356    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:49:39.491265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:39.491234    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl"]
Nov 02 21:49:40.603317 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:40.603279    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="573a1a40-d306-4239-b093-79fe496a41eb" path="/var/lib/kubelet/pods/573a1a40-d306-4239-b093-79fe496a41eb/volumes"
Nov 02 21:49:40.604001 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:40.603983    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84744649-e77a-47bd-a63d-eddfb0961762" path="/var/lib/kubelet/pods/84744649-e77a-47bd-a63d-eddfb0961762/volumes"
Nov 02 21:49:40.604603 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:40.604586    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" path="/var/lib/kubelet/pods/857bf1d9-a6b8-4d14-908e-8669c9647bef/volumes"
Nov 02 21:49:42.460714 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.460683    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:49:42.461286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461270    4888 cpu_manager.go:414] "containerMap: removing container" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.461330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461287    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.461330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461303    4888 cpu_manager.go:414] "containerMap: removing container" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy"
Nov 02 21:49:42.461330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461309    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy"
Nov 02 21:49:42.461330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461322    4888 cpu_manager.go:414] "containerMap: removing container" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.461330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461327    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.461472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461348    4888 cpu_manager.go:414] "containerMap: removing container" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy"
Nov 02 21:49:42.461472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461352    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy"
Nov 02 21:49:42.461472 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461458    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="573a1a40-d306-4239-b093-79fe496a41eb" containerName="istio-proxy"
Nov 02 21:49:42.461555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461472    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.461555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461481    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="857bf1d9-a6b8-4d14-908e-8669c9647bef" containerName="istio-proxy"
Nov 02 21:49:42.461555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.461488    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="84744649-e77a-47bd-a63d-eddfb0961762" containerName="istio-proxy"
Nov 02 21:49:42.466565 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.466544    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619204    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619256    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619341    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619414    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619444    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqthj\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619527    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619629    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.619870 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.619812    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.705132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.705093    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:49:42.721101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.721017    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.721252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.721229    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.721298 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.721263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722080    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722181    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722223    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722258    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqthj\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722299    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722332    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.722424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722413    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.723295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722595    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.723295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.722779    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.723295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.723157    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.723461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.723366    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.725700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.725657    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.726038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.726012    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.878880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.850686    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:42.902880 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:42.901286    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqthj\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\") pod \"e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") " pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:43.082744 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:43.082707    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:43.511797 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:49:43.511763    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c71dad7_7040_40d6_8f68_c83b010a8c95.slice/crio-51d8a3a71ae2225f13f0ca7f8b98fc20bfbe38138b27241e19dd37c842d35133 WatchSource:0}: Error finding container 51d8a3a71ae2225f13f0ca7f8b98fc20bfbe38138b27241e19dd37c842d35133: Status 404 returned error can't find the container with id 51d8a3a71ae2225f13f0ca7f8b98fc20bfbe38138b27241e19dd37c842d35133
Nov 02 21:49:43.512251 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:43.511994    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:49:43.514660 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:43.514632    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:43.514726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:43.514690    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:43.514726 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:43.514720    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:49:44.151088 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:44.151046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" event={"ID":"3c71dad7-7040-40d6-8f68-c83b010a8c95","Type":"ContainerStarted","Data":"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"}
Nov 02 21:49:44.151288 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:44.151121    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" event={"ID":"3c71dad7-7040-40d6-8f68-c83b010a8c95","Type":"ContainerStarted","Data":"51d8a3a71ae2225f13f0ca7f8b98fc20bfbe38138b27241e19dd37c842d35133"}
Nov 02 21:49:44.323872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:44.320442    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podStartSLOduration=2.320421708 podStartE2EDuration="2.320421708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:49:44.309300554 +0000 UTC m=+2213.853092471" watchObservedRunningTime="2026-11-02 21:49:44.320421708 +0000 UTC m=+2213.864213625"
Nov 02 21:49:45.087020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:45.086980    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:45.087809 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:45.087780    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:45.088016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:45.087991    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:46.083917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:46.083866    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:46.084115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:46.083944    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:47.083864 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:47.083821    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:47.084277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:47.083913    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:48.084041 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:48.084003    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:48.084424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:48.084060    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:49.083485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:49.083447    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:49.083691 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:49.083511    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:50.083752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:50.083235    4888 patch_prober.go:29] interesting pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 02 21:49:50.083752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:50.083295    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 02 21:49:51.089019 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:51.088986    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:51.089445 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:51.089432    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:51.090185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:49:51.090166    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:49:52.279433 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:49:52.279396    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:50:05.489478 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:05.489442    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:50:05.490119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:05.489710    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy" containerID="cri-o://1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533" gracePeriod=30
Nov 02 21:50:06.847693 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:06.847653    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 21:50:06.853572 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:06.853546    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:06.908164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:06.908120    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c1243c-openshift-default-dockercfg-sdlzj\""
Nov 02 21:50:06.987882 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:06.981038    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 21:50:07.023497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.023451    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.023763 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.023737    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.023939 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.023920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.035993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.035953    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.036276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.036259    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.036406 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.036391    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.038300 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.036667    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.038682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.038658    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v447\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.038791 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.038706    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.173945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.142280    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v447\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.181910 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.174220    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.199919 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.182330    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.200243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.200203    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.200469 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.200451    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.200655 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.200630    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.200899 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.200877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.201049 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.201033    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.201256 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.201242    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.249835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.202684    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.250150 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.207384    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.250276 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.179444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.250371 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.183199    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.250451 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.207958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.250530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.219275    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v447\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.252163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.252034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.252992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.252921    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.277740 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.276041    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\") pod \"e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") " pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.469302 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.469213    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:07.898335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.898310    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 21:50:07.900300 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:50:07.900272    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da2cc5d_90d8_4a6f_9662_6e9b1ca4de66.slice/crio-560eeb2c4ed3c30983386cfe64f786585d76f93fec4a4f37fd0e6feafb0c17c7 WatchSource:0}: Error finding container 560eeb2c4ed3c30983386cfe64f786585d76f93fec4a4f37fd0e6feafb0c17c7: Status 404 returned error can't find the container with id 560eeb2c4ed3c30983386cfe64f786585d76f93fec4a4f37fd0e6feafb0c17c7
Nov 02 21:50:07.903007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.902973    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:07.903075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.903044    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:07.903075 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:07.903073    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:08.340201 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:08.340161    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerStarted","Data":"e292c25f64d24d362907cac5cf72b1458754a6a46300e380fce5eebdeede4f71"}
Nov 02 21:50:08.340408 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:08.340225    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerStarted","Data":"560eeb2c4ed3c30983386cfe64f786585d76f93fec4a4f37fd0e6feafb0c17c7"}
Nov 02 21:50:08.469473 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:08.469433    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:08.475776 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:08.475745    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:08.499551 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:08.499496    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podStartSLOduration=2.4994815519999998 podStartE2EDuration="2.499481552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:08.449033066 +0000 UTC m=+2237.992824996" watchObservedRunningTime="2026-11-02 21:50:08.499481552 +0000 UTC m=+2238.043273504"
Nov 02 21:50:09.346589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:09.346555    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:09.347591 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:09.347570    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:50:10.591368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.591330    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:50:10.591834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.591532    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" containerID="cri-o://1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187" gracePeriod=30
Nov 02 21:50:10.788230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.788205    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:50:10.877194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877163    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877217    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877293    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877412 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877387    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877440    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877452    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:10.877582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877511    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877600    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877631    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:10.877736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877653    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.877736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877664    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:10.877736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877708    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\") pod \"baa99491-e8a5-4328-8bf6-2f2dc5446954\" (UID: \"baa99491-e8a5-4328-8bf6-2f2dc5446954\") "
Nov 02 21:50:10.878027 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.877862    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:10.878079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878044    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:10.878295 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878277    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.878385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878299    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.878385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878314    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.878385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878327    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.878385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.878341    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa99491-e8a5-4328-8bf6-2f2dc5446954-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.879790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.879763    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "kube-api-access-9mjtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:10.879972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.879932    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:10.880108 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.880091    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:10.880211 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.880192    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy" pod "baa99491-e8a5-4328-8bf6-2f2dc5446954" (UID: "baa99491-e8a5-4328-8bf6-2f2dc5446954"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:10.980074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.980038    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mjtj\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-kube-api-access-9mjtj\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.980074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.980072    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.980074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.980082    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:10.980342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:10.980091    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa99491-e8a5-4328-8bf6-2f2dc5446954-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.359555 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.359505    4888 generic.go:356] "Generic (PLEG): container finished" podID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerID="1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533" exitCode=0
Nov 02 21:50:11.359760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.359599    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"
Nov 02 21:50:11.359760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.359637    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerDied","Data":"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"}
Nov 02 21:50:11.359760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.359733    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf" event={"ID":"baa99491-e8a5-4328-8bf6-2f2dc5446954","Type":"ContainerDied","Data":"fa02a9a3ed18597a986a3c1c64c4542b10902cf3b527d6627498d9c1fe5bd1b7"}
Nov 02 21:50:11.359990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.359762    4888 scope.go:122] "RemoveContainer" containerID="1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"
Nov 02 21:50:11.361710 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.361686    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:50:11.361833 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.361735    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:50:11.371385 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.371361    4888 scope.go:122] "RemoveContainer" containerID="939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"
Nov 02 21:50:11.381433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.381410    4888 scope.go:122] "RemoveContainer" containerID="1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"
Nov 02 21:50:11.381807 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:11.381782    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533\": container with ID starting with 1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533 not found: ID does not exist" containerID="1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"
Nov 02 21:50:11.381924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.381816    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533"} err="failed to get container status \"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533\": rpc error: code = NotFound desc = could not find container \"1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533\": container with ID starting with 1a18bae63a14ded0e70af60ccd607e9a15a885834caaaffce54f0db43c299533 not found: ID does not exist"
Nov 02 21:50:11.381924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.381860    4888 scope.go:122] "RemoveContainer" containerID="939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"
Nov 02 21:50:11.382161 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:11.382140    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409\": container with ID starting with 939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409 not found: ID does not exist" containerID="939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"
Nov 02 21:50:11.382215 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.382169    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409"} err="failed to get container status \"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409\": rpc error: code = NotFound desc = could not find container \"939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409\": container with ID starting with 939835f7292fe2029d4aba8b1ad45eb6a569b92a90f5b70b48f00cabc2929409 not found: ID does not exist"
Nov 02 21:50:11.520518 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.520473    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:50:11.570686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.570499    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf"]
Nov 02 21:50:11.575682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.575499    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:50:11.575871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:11.575744    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" containerID="cri-o://a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0" gracePeriod=30
Nov 02 21:50:12.602834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:12.602786    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" path="/var/lib/kubelet/pods/baa99491-e8a5-4328-8bf6-2f2dc5446954/volumes"
Nov 02 21:50:13.482900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:13.481993    4888 patch_prober.go:29] interesting pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:50:13.482900 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:13.482080    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:50:15.876051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.876021    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:50:15.911210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911170    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911211    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911234    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911274    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911336    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911368    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911413    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911454    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911538    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\") pod \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\" (UID: \"ba80d8e6-2176-4023-b009-e9da7e6c9c83\") "
Nov 02 21:50:15.911790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911544    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:15.911790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911622    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:15.911790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911632    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:15.912059 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.911925    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:15.912153 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.912125    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:15.912280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.912161    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:15.912280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.912181    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:15.912280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.912195    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:15.912280 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.912209    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:15.913835 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.913811    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:15.913944 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.913875    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:15.914002 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.913988    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:15.914387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:15.914361    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd" pod "ba80d8e6-2176-4023-b009-e9da7e6c9c83" (UID: "ba80d8e6-2176-4023-b009-e9da7e6c9c83"). InnerVolumeSpecName "kube-api-access-g7cmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:16.014073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.014034    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.014073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.014067    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.014073 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.014076    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.014397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.014086    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba80d8e6-2176-4023-b009-e9da7e6c9c83-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.014397 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.014095    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7cmd\" (UniqueName: \"kubernetes.io/projected/ba80d8e6-2176-4023-b009-e9da7e6c9c83-kube-api-access-g7cmd\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.417813 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.417779    4888 generic.go:356] "Generic (PLEG): container finished" podID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerID="1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187" exitCode=0
Nov 02 21:50:16.418018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.417839    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"
Nov 02 21:50:16.418018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.417869    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" event={"ID":"ba80d8e6-2176-4023-b009-e9da7e6c9c83","Type":"ContainerDied","Data":"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"}
Nov 02 21:50:16.418018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.417943    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd" event={"ID":"ba80d8e6-2176-4023-b009-e9da7e6c9c83","Type":"ContainerDied","Data":"21e28e0225c520de70215af2bf1a792032bf9d8a3e1d2bff320ef6749838082d"}
Nov 02 21:50:16.418018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.417974    4888 scope.go:122] "RemoveContainer" containerID="1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"
Nov 02 21:50:16.429790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.429770    4888 scope.go:122] "RemoveContainer" containerID="1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"
Nov 02 21:50:16.430131 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:16.430106    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187\": container with ID starting with 1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187 not found: ID does not exist" containerID="1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"
Nov 02 21:50:16.430179 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.430146    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187"} err="failed to get container status \"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187\": rpc error: code = NotFound desc = could not find container \"1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187\": container with ID starting with 1943a7d5b0c277432d0646a59665aa0ed8dba1453e4170b9e382ded2e1f15187 not found: ID does not exist"
Nov 02 21:50:16.579756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.579716    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:50:16.627220 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.627185    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd"]
Nov 02 21:50:16.866446 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.866417    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:50:16.924057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924024    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924085    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924118    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924138    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924239    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924275    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924312    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924347    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924373    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:16.924527 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924412    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\") pod \"150eec8f-a279-4e86-89ee-d9db889d304f\" (UID: \"150eec8f-a279-4e86-89ee-d9db889d304f\") "
Nov 02 21:50:16.925063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924635    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:16.925063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924725    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:16.925063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924789    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:16.925063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.924929    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:16.925257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.925131    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.925257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.925152    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/150eec8f-a279-4e86-89ee-d9db889d304f-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.925257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.925168    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.925257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.925184    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.925257 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.925198    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:16.926610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.926583    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:16.926610 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.926597    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:16.926903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.926885    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:16.926990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:16.926969    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p" pod "150eec8f-a279-4e86-89ee-d9db889d304f" (UID: "150eec8f-a279-4e86-89ee-d9db889d304f"). InnerVolumeSpecName "kube-api-access-k4m2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:17.026509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.026424    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/150eec8f-a279-4e86-89ee-d9db889d304f-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:17.026509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.026456    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:17.026509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.026466    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4m2p\" (UniqueName: \"kubernetes.io/projected/150eec8f-a279-4e86-89ee-d9db889d304f-kube-api-access-k4m2p\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:17.026509 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.026476    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/150eec8f-a279-4e86-89ee-d9db889d304f-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:17.103992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.103949    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:50:17.426904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.426865    4888 generic.go:356] "Generic (PLEG): container finished" podID="150eec8f-a279-4e86-89ee-d9db889d304f" containerID="a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0" exitCode=0
Nov 02 21:50:17.427089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.426912    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"
Nov 02 21:50:17.427089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.426961    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerDied","Data":"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"}
Nov 02 21:50:17.427089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.427018    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq" event={"ID":"150eec8f-a279-4e86-89ee-d9db889d304f","Type":"ContainerDied","Data":"d75c9bced9c95c4a51d6b47f811ba756acaa6f7af31afc959d4054d23e10b0e1"}
Nov 02 21:50:17.427089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.427046    4888 scope.go:122] "RemoveContainer" containerID="a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"
Nov 02 21:50:17.438349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.438322    4888 scope.go:122] "RemoveContainer" containerID="26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"
Nov 02 21:50:17.447735 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.447711    4888 scope.go:122] "RemoveContainer" containerID="a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"
Nov 02 21:50:17.448084 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:17.448064    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0\": container with ID starting with a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0 not found: ID does not exist" containerID="a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"
Nov 02 21:50:17.448131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.448094    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0"} err="failed to get container status \"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0\": rpc error: code = NotFound desc = could not find container \"a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0\": container with ID starting with a19dfb25e1cdf94e206b2b2b8667bace1749e437097a8869cc1995c4041631a0 not found: ID does not exist"
Nov 02 21:50:17.448131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.448115    4888 scope.go:122] "RemoveContainer" containerID="26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"
Nov 02 21:50:17.448365 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:17.448346    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba\": container with ID starting with 26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba not found: ID does not exist" containerID="26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"
Nov 02 21:50:17.448424 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.448377    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba"} err="failed to get container status \"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba\": rpc error: code = NotFound desc = could not find container \"26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba\": container with ID starting with 26f8f8bc0e3aabc49bc04b43aebd7a395cf10ed756a465a44621397910edf1ba not found: ID does not exist"
Nov 02 21:50:17.583943 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.583900    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:50:17.589236 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:17.589204    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq"]
Nov 02 21:50:18.251834 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.251783    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 21:50:18.257592 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.257558    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy"
Nov 02 21:50:18.257790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.257776    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy"
Nov 02 21:50:18.257982 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.257965    4888 cpu_manager.go:414] "containerMap: removing container" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.258089 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258074    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.258170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258161    4888 cpu_manager.go:414] "containerMap: removing container" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.258246 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258236    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.258525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258512    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.258616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258606    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.258707 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.258697    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" containerName="istio-proxy"
Nov 02 21:50:18.260187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.260157    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.260348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.260339    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.287871 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.277098    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.313025 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.312993    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 21:50:18.329324 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.329277    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c1243c-dockercfg-62p7c\""
Nov 02 21:50:18.347203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.345125    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c1243c\""
Nov 02 21:50:18.366311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.365835    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.366311 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.365981    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.435275 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.435235    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 21:50:18.436214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.436187    4888 cpu_manager.go:414] "containerMap: removing container" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.436329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.436218    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baa99491-e8a5-4328-8bf6-2f2dc5446954" containerName="istio-proxy"
Nov 02 21:50:18.436329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.436236    4888 cpu_manager.go:414] "containerMap: removing container" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.436329 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.436245    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" containerName="istio-proxy"
Nov 02 21:50:18.444669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.444637    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.462168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.462133    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 21:50:18.467410 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.467375    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.467807 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.467780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.467962 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.467940    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfpl\" (UniqueName: \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.468051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.468037    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.469503 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.469463    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.522921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.522805    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.569594 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.569562    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfpl\" (UniqueName: \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.569760 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.569665    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.570409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.570382    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.602477 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.602441    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="150eec8f-a279-4e86-89ee-d9db889d304f" path="/var/lib/kubelet/pods/150eec8f-a279-4e86-89ee-d9db889d304f/volumes"
Nov 02 21:50:18.603050 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.603032    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba80d8e6-2176-4023-b009-e9da7e6c9c83" path="/var/lib/kubelet/pods/ba80d8e6-2176-4023-b009-e9da7e6c9c83/volumes"
Nov 02 21:50:18.623632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.623600    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfpl\" (UniqueName: \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\") pod \"payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:18.644589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.644542    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:18.770217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:18.770183    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:19.019138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.019104    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 21:50:19.019942 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:50:19.019911    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1cc552_3f30_40b9_a564_01fca3fe3440.slice/crio-0b210827763ea5a3edc79ca8c532fdb9eddb83c0d11832f6fa13e414395d3a27 WatchSource:0}: Error finding container 0b210827763ea5a3edc79ca8c532fdb9eddb83c0d11832f6fa13e414395d3a27: Status 404 returned error can't find the container with id 0b210827763ea5a3edc79ca8c532fdb9eddb83c0d11832f6fa13e414395d3a27
Nov 02 21:50:19.142676 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.142640    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 21:50:19.143721 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:50:19.143693    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e03ac2_be04_4d3e_ac44_f319bfe2e789.slice/crio-ffed1c6eeacf33dd981a44c6d66c813a6275237b510e112184c0ef99bf1ea4ff WatchSource:0}: Error finding container ffed1c6eeacf33dd981a44c6d66c813a6275237b510e112184c0ef99bf1ea4ff: Status 404 returned error can't find the container with id ffed1c6eeacf33dd981a44c6d66c813a6275237b510e112184c0ef99bf1ea4ff
Nov 02 21:50:19.453747 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.453710    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" event={"ID":"13e03ac2-be04-4d3e-ac44-f319bfe2e789","Type":"ContainerStarted","Data":"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"}
Nov 02 21:50:19.454233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.453767    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" event={"ID":"13e03ac2-be04-4d3e-ac44-f319bfe2e789","Type":"ContainerStarted","Data":"ffed1c6eeacf33dd981a44c6d66c813a6275237b510e112184c0ef99bf1ea4ff"}
Nov 02 21:50:19.454233 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.453799    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:19.455522 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.455495    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" event={"ID":"2b1cc552-3f30-40b9-a564-01fca3fe3440","Type":"ContainerStarted","Data":"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"}
Nov 02 21:50:19.455645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.455540    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" event={"ID":"2b1cc552-3f30-40b9-a564-01fca3fe3440","Type":"ContainerStarted","Data":"0b210827763ea5a3edc79ca8c532fdb9eddb83c0d11832f6fa13e414395d3a27"}
Nov 02 21:50:19.455645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.455577    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:19.562416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.562361    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" podStartSLOduration=1.562347982 podStartE2EDuration="1.562347982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:19.560917003 +0000 UTC m=+2249.104708919" watchObservedRunningTime="2026-11-02 21:50:19.562347982 +0000 UTC m=+2249.106139897"
Nov 02 21:50:19.668980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:19.668926    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" podStartSLOduration=1.668913405 podStartE2EDuration="1.668913405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:19.667646433 +0000 UTC m=+2249.211438348" watchObservedRunningTime="2026-11-02 21:50:19.668913405 +0000 UTC m=+2249.212705320"
Nov 02 21:50:22.261039 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:22.261002    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:50:30.465715 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:30.465677    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 21:50:30.466265 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:30.465765    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 21:50:32.126195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.126126    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main" containerID="cri-o://fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302" gracePeriod=45
Nov 02 21:50:32.392650 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.392580    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:50:32.532467 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532431    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.532667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532572    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.532667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532600    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\" (UniqueName: \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.532667 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532641    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.532826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532701    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.532826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532704    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:32.532826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.532764    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\") pod \"0ba31a8d-ca21-48d1-b13f-236423e3015b\" (UID: \"0ba31a8d-ca21-48d1-b13f-236423e3015b\") "
Nov 02 21:50:32.533114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.533058    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:32.533180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.533142    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:32.533416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.533390    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.533566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.533441    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.533566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.533458    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.535195 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.535166    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:50:32.535296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.535243    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "kube-api-access-rl5tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:32.535296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.535268    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm" pod "0ba31a8d-ca21-48d1-b13f-236423e3015b" (UID: "0ba31a8d-ca21-48d1-b13f-236423e3015b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:32.541525 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.541489    4888 generic.go:356] "Generic (PLEG): container finished" podID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerID="fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302" exitCode=0
Nov 02 21:50:32.541628 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.541530    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"
Nov 02 21:50:32.541628 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.541547    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" event={"ID":"0ba31a8d-ca21-48d1-b13f-236423e3015b","Type":"ContainerDied","Data":"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"}
Nov 02 21:50:32.541628 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.541589    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb" event={"ID":"0ba31a8d-ca21-48d1-b13f-236423e3015b","Type":"ContainerDied","Data":"e9897d6d337fa6c32a868c9ed9597721732e9abca7975dd3a492c908508625f8"}
Nov 02 21:50:32.541737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.541637    4888 scope.go:122] "RemoveContainer" containerID="fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"
Nov 02 21:50:32.563357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.563322    4888 scope.go:122] "RemoveContainer" containerID="fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"
Nov 02 21:50:32.563721 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:32.563693    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302\": container with ID starting with fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302 not found: ID does not exist" containerID="fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"
Nov 02 21:50:32.563799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.563730    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302"} err="failed to get container status \"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302\": rpc error: code = NotFound desc = could not find container \"fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302\": container with ID starting with fc1b1740a30a692964514116c845d093a80ef5630739ec20d50252a58830f302 not found: ID does not exist"
Nov 02 21:50:32.634571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.634534    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba31a8d-ca21-48d1-b13f-236423e3015b-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.634571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.634570    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl5tm\" (UniqueName: \"kubernetes.io/projected/0ba31a8d-ca21-48d1-b13f-236423e3015b-kube-api-access-rl5tm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.634826 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.634586    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba31a8d-ca21-48d1-b13f-236423e3015b-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:50:32.691537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.691497    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:50:32.695919 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:32.695881    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-dd6574/distinct-w0-dd6574-kserve-77b8668c5-5w9qb"]
Nov 02 21:50:34.602732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:34.602698    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" path="/var/lib/kubelet/pods/0ba31a8d-ca21-48d1-b13f-236423e3015b/volumes"
Nov 02 21:50:46.844724 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:46.844689    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 21:50:46.845303 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:46.845284    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main"
Nov 02 21:50:46.845303 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:46.845297    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main"
Nov 02 21:50:46.845429 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:46.845412    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ba31a8d-ca21-48d1-b13f-236423e3015b" containerName="main"
Nov 02 21:50:46.848733 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:46.848712    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.049867 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.049157    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3566f9-openshift-default-dockercfg-snfbv\""
Nov 02 21:50:47.071877 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.071823    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 21:50:47.088818 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.088763    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.089202 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.089164    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.089768 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.089555    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.089953    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.102042    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zljts\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.102116    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.102182    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.102212    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.104270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.102284    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.208435 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.208245    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.208711 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.208679    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.208824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.208753    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.208824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.208810    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.209007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.208986    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.209067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.209390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209368    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209450    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zljts\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209528    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209613    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209658    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.209732    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210095 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.210091    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.210545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.210315    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.213343 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.213317    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.213506 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.213398    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.264398 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.264355    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.264567 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.264444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zljts\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\") pod \"e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") " pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.463430 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.463392    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:47.959892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.952839    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 21:50:47.969886 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:50:47.962579    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851f0db2_7ab7_477a_95e9_310d2516314a.slice/crio-c72740af528e40dad0d08c17c6f71f969d29e0b7d43a71c1eb7e3f05d6c651d2 WatchSource:0}: Error finding container c72740af528e40dad0d08c17c6f71f969d29e0b7d43a71c1eb7e3f05d6c651d2: Status 404 returned error can't find the container with id c72740af528e40dad0d08c17c6f71f969d29e0b7d43a71c1eb7e3f05d6c651d2
Nov 02 21:50:47.994520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.994239    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:47.995536 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.995499    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:47.998139 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:47.998093    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:50:48.663072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:48.663027    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerStarted","Data":"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"}
Nov 02 21:50:48.663286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:48.663093    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerStarted","Data":"c72740af528e40dad0d08c17c6f71f969d29e0b7d43a71c1eb7e3f05d6c651d2"}
Nov 02 21:50:48.777677 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:48.777615    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podStartSLOduration=2.777600398 podStartE2EDuration="2.777600398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:48.775789589 +0000 UTC m=+2278.319581506" watchObservedRunningTime="2026-11-02 21:50:48.777600398 +0000 UTC m=+2278.321392313"
Nov 02 21:50:49.464198 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:49.464160    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:49.470038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:49.470001    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:49.669497 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:49.669469    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:49.670668 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:50:49.670645    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:50:52.259911 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:50:52.259802    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:51:01.989485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:01.989391    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:51:17.011039 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.010973    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main" containerID="cri-o://af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2" gracePeriod=45
Nov 02 21:51:17.363154 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.363122    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:51:17.438717 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.438681    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.438945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.438802    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.438945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.438929    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.439088 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.438987    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\" (UniqueName: \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.439088 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439047    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.439204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439106    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:51:17.439204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439178    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\") pod \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\" (UID: \"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d\") "
Nov 02 21:51:17.439342 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439318    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:51:17.439922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439843    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.439922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439883    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.439922 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.439887    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:51:17.441918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.441878    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:51:17.442132 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.442104    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:51:17.442213 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.442136    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn" pod "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" (UID: "ac5a6da6-b24e-44de-8ce5-2b68cc40a49d"). InnerVolumeSpecName "kube-api-access-sfgqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:51:17.541990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.541883    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.541990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.541920    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.541990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.541937    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfgqn\" (UniqueName: \"kubernetes.io/projected/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-kube-api-access-sfgqn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.541990 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.541953    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:17.885978 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.885934    4888 generic.go:356] "Generic (PLEG): container finished" podID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerID="af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2" exitCode=0
Nov 02 21:51:17.886183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.885982    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"
Nov 02 21:51:17.886183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.886016    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" event={"ID":"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d","Type":"ContainerDied","Data":"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"}
Nov 02 21:51:17.886183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.886070    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj" event={"ID":"ac5a6da6-b24e-44de-8ce5-2b68cc40a49d","Type":"ContainerDied","Data":"59cd3d6dbabbd8aeb66762de3c0152ddec9eaed3609e7d1ed9ad75f9ff72eddb"}
Nov 02 21:51:17.886183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.886092    4888 scope.go:122] "RemoveContainer" containerID="af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"
Nov 02 21:51:17.898588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.898569    4888 scope.go:122] "RemoveContainer" containerID="af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"
Nov 02 21:51:17.899120 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:51:17.899096    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2\": container with ID starting with af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2 not found: ID does not exist" containerID="af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"
Nov 02 21:51:17.899302 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:17.899268    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2"} err="failed to get container status \"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2\": rpc error: code = NotFound desc = could not find container \"af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2\": container with ID starting with af5cafd9e689d7a3e0946fcdca4f18d017e83f48b29963fdfd43bb643ca53ab2 not found: ID does not exist"
Nov 02 21:51:18.042032 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:18.041990    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:51:18.044980 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:18.044951    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6b983e02/e2e-lifecycle-model-6b983e02-kserve-5d9cc5b55c-sqvdj"]
Nov 02 21:51:18.602057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:18.602015    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" path="/var/lib/kubelet/pods/ac5a6da6-b24e-44de-8ce5-2b68cc40a49d/volumes"
Nov 02 21:51:19.901945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.901862    4888 generic.go:356] "Generic (PLEG): container finished" podID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerID="56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3" exitCode=0
Nov 02 21:51:19.901945 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.901926    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerDied","Data":"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"}
Nov 02 21:51:19.902431 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.902388    4888 scope.go:122] "RemoveContainer" containerID="56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"
Nov 02 21:51:19.904121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.904095    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:19.904187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.904168    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:19.904234 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:19.904209    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:20.937894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:20.937810    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerStarted","Data":"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"}
Nov 02 21:51:21.775635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:21.775596    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:51:21.777134 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:21.777103    4888 patch_prober.go:29] interesting pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 02 21:51:21.777247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:21.777178    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 02 21:51:22.073772 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:22.073686    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:51:22.262730 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:51:22.262683    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:51:22.776690 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:22.776647    4888 patch_prober.go:29] interesting pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 02 21:51:22.776927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:22.776719    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 02 21:51:23.776231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:23.776195    4888 patch_prober.go:29] interesting pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 02 21:51:23.776643 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:23.776252    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 02 21:51:24.776453 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:24.776414    4888 patch_prober.go:29] interesting pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 02 21:51:24.776925 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:24.776487    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 02 21:51:25.775843 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:25.775804    4888 patch_prober.go:29] interesting pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 02 21:51:25.776068 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:25.775894    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 02 21:51:26.782420 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:26.782385    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:51:26.783615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:26.783591    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:51:33.196740 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.196696    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:51:33.197395 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.196994    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" podUID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" containerName="maas-api" containerID="cri-o://866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a" gracePeriod=30
Nov 02 21:51:33.587653 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.587346    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:51:33.734485 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.734438    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\") pod \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") "
Nov 02 21:51:33.734921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.734898    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\" (UniqueName: \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\") pod \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") "
Nov 02 21:51:33.735213 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.735176    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\") pod \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") "
Nov 02 21:51:33.735388 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.735364    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") pod \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\" (UID: \"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe\") "
Nov 02 21:51:33.737055 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.736700    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca" pod "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" (UID: "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:51:33.766783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.766748    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls" pod "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" (UID: "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:51:33.774284 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.774199    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs" pod "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" (UID: "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:51:33.774625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.774579    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc" pod "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" (UID: "0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe"). InnerVolumeSpecName "kube-api-access-skvmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:51:33.837107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.837020    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:33.837107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.837055    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:33.837107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.837064    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skvmc\" (UniqueName: \"kubernetes.io/projected/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-kube-api-access-skvmc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:33.837107 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:33.837075    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:51:34.031101 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.031063    4888 generic.go:356] "Generic (PLEG): container finished" podID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" containerID="866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a" exitCode=0
Nov 02 21:51:34.031310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.031113    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"
Nov 02 21:51:34.031310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.031147    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" event={"ID":"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe","Type":"ContainerDied","Data":"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"}
Nov 02 21:51:34.031310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.031211    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn" event={"ID":"0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe","Type":"ContainerDied","Data":"c69b4c34ac0d3d7713dfcb6e3cb3d735267cec0804260a92d01fb667fde81a74"}
Nov 02 21:51:34.031310 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.031244    4888 scope.go:122] "RemoveContainer" containerID="866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"
Nov 02 21:51:34.041921 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.041895    4888 scope.go:122] "RemoveContainer" containerID="866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"
Nov 02 21:51:34.042264 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:51:34.042243    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a\": container with ID starting with 866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a not found: ID does not exist" containerID="866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"
Nov 02 21:51:34.042312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.042274    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a"} err="failed to get container status \"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a\": rpc error: code = NotFound desc = could not find container \"866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a\": container with ID starting with 866ac73c47faa1022d7b028bf20979c9517691ad7cf3686a6a409b456cb0fe0a not found: ID does not exist"
Nov 02 21:51:34.202873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.198761    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:51:34.245004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.244959    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dd6574-d6845899f-pzgbn"]
Nov 02 21:51:34.601480 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:34.601394    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" path="/var/lib/kubelet/pods/0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe/volumes"
Nov 02 21:51:48.172955 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.172331    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 21:51:48.176817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.176789    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" containerName="maas-api"
Nov 02 21:51:48.177009 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.176996    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" containerName="maas-api"
Nov 02 21:51:48.177072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.177066    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main"
Nov 02 21:51:48.177122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.177115    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main"
Nov 02 21:51:48.177319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.177310    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f86cd50-af04-43bd-b5b9-9e1ad99c3dbe" containerName="maas-api"
Nov 02 21:51:48.177375 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.177369    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac5a6da6-b24e-44de-8ce5-2b68cc40a49d" containerName="main"
Nov 02 21:51:48.182682 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.182529    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.231327 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.231290    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 21:51:48.282611 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282570    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.282790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282624    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.282790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282751    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.282913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282798    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.282913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282816    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.282913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.282904    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.283037 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.283008    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.283143 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.283130    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt8vs\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.283182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.283170    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.287738 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.287708    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3d2b29-openshift-default-dockercfg-ct4l8\""
Nov 02 21:51:48.384732 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.384696    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt8vs\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385077 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385053    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385264 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385243    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385426 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385408    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385611    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385767    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.385920 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.385892    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386062 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386044    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386186 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386163    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386195    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386312 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386426    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.386626 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386601    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.387002 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.386981    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.388523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.388498    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.389384 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.389362    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.450872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.450779    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt8vs\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.457038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.456973    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\") pod \"e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") " pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.503686 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.503650    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:48.929670 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.929638    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 21:51:48.930449 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:51:48.930425    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bfd517_4735_4472_8a0a_963f57391648.slice/crio-78f794da366d68f65ab9cf791fc7d2ca95c7fd0c003e1a153a594aa551bb6d1c WatchSource:0}: Error finding container 78f794da366d68f65ab9cf791fc7d2ca95c7fd0c003e1a153a594aa551bb6d1c: Status 404 returned error can't find the container with id 78f794da366d68f65ab9cf791fc7d2ca95c7fd0c003e1a153a594aa551bb6d1c
Nov 02 21:51:48.933028 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.932995    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:48.933119 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.933075    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:48.933194 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:48.933120    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:51:49.145665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:49.145624    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerStarted","Data":"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"}
Nov 02 21:51:49.145884 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:49.145706    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerStarted","Data":"78f794da366d68f65ab9cf791fc7d2ca95c7fd0c003e1a153a594aa551bb6d1c"}
Nov 02 21:51:49.257523 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:49.257397    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podStartSLOduration=1.25737841 podStartE2EDuration="1.25737841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:49.255940371 +0000 UTC m=+2338.799732287" watchObservedRunningTime="2026-11-02 21:51:49.25737841 +0000 UTC m=+2338.801170327"
Nov 02 21:51:49.503903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:49.503862    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:49.511227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:49.511150    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:50.154588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:50.154552    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:50.155813 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:50.155786    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:51:52.260503 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:51:52.260463    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:51:56.181105 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.181058    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:51:56.272004 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.271973    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:51:56.272180 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.272093    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.335020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.334981    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-3d2b29\""
Nov 02 21:51:56.365941 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.365901    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9wl\" (UniqueName: \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.366131 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.366065    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.381895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.381840    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-3d2b29-dockercfg-lm7l6\""
Nov 02 21:51:56.467071 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.466970    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9wl\" (UniqueName: \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.467274 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.467129    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.467913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.467894    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.536289 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.536252    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9wl\" (UniqueName: \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\") pod \"payload-processing-e2e-api-b-3d2b29-568d868777-xbm67\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") " pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.592409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.592371    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:56.989464 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:56.989258    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:51:56.989579 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:51:56.989510    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff3bdee_9072_4a1e_b88b_86e20015c001.slice/crio-b6767d5334d323ce18fe2dc9ca9e99d261b70d04c8dd48ac165b3f3becbbedc7 WatchSource:0}: Error finding container b6767d5334d323ce18fe2dc9ca9e99d261b70d04c8dd48ac165b3f3becbbedc7: Status 404 returned error can't find the container with id b6767d5334d323ce18fe2dc9ca9e99d261b70d04c8dd48ac165b3f3becbbedc7
Nov 02 21:51:57.284762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:57.284672    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" event={"ID":"aff3bdee-9072-4a1e-b88b-86e20015c001","Type":"ContainerStarted","Data":"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"}
Nov 02 21:51:57.284762 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:57.284729    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" event={"ID":"aff3bdee-9072-4a1e-b88b-86e20015c001","Type":"ContainerStarted","Data":"b6767d5334d323ce18fe2dc9ca9e99d261b70d04c8dd48ac165b3f3becbbedc7"}
Nov 02 21:51:57.285207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:57.284769    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:51:57.397687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:51:57.397627    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" podStartSLOduration=1.397612577 podStartE2EDuration="1.397612577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:57.395837976 +0000 UTC m=+2346.939629893" watchObservedRunningTime="2026-11-02 21:51:57.397612577 +0000 UTC m=+2346.941404493"
Nov 02 21:52:03.326351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.326246    4888 generic.go:356] "Generic (PLEG): container finished" podID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerID="e292c25f64d24d362907cac5cf72b1458754a6a46300e380fce5eebdeede4f71" exitCode=0
Nov 02 21:52:03.326351 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.326318    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerDied","Data":"e292c25f64d24d362907cac5cf72b1458754a6a46300e380fce5eebdeede4f71"}
Nov 02 21:52:03.326938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.326752    4888 scope.go:122] "RemoveContainer" containerID="e292c25f64d24d362907cac5cf72b1458754a6a46300e380fce5eebdeede4f71"
Nov 02 21:52:03.328416 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.328383    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:03.328517 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.328456    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:03.328517 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:03.328491    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:04.340888 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:04.340827    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerStarted","Data":"bcbe74cc0205c4b43c69c3372b7eab7331a7e57c0f0e14ffd7839955474bedcf"}
Nov 02 21:52:04.471053 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:04.471006    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:52:04.471896 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:04.471842    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:04.472171 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:04.472143    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:05.470396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:05.470357    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:05.470799 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:05.470415    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:06.470436 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:06.470395    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:06.470916 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:06.470467    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:07.470590 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:07.470547    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:07.471038 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:07.470618    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:08.316876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.312051    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:52:08.502207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.502173    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:08.555914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.555874    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:08.878249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.878206    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:52:08.878520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.878291    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:52:08.878520 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.878328    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:52:08.879120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.879001    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy" containerID="cri-o://0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c" gracePeriod=30
Nov 02 21:52:08.879419 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.879388    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy" containerID="cri-o://97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f" gracePeriod=30
Nov 02 21:52:08.881270 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:08.881237    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy" containerID="cri-o://4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236" gracePeriod=30
Nov 02 21:52:09.346504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:09.346465    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:52:09.471296 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:09.471259    4888 patch_prober.go:29] interesting pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused" start-of-body=
Nov 02 21:52:09.472669 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:09.472632    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.131:15021/healthz/ready\": dial tcp 10.132.0.131:15021: connect: connection refused"
Nov 02 21:52:10.474968 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:10.474935    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:52:10.476200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:10.476180    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 21:52:14.428110 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.428076    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:52:14.482297 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.482272    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:52:14.505876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.505773    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.505876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.505833    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.505906    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.505956    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506003    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506051    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506116    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506216 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506166    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.506301 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506284    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\") pod \"d36e86b4-ae71-4466-82c8-6412967f2836\" (UID: \"d36e86b4-ae71-4466-82c8-6412967f2836\") "
Nov 02 21:52:14.507067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506937    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.507067 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.506953    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.507206 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.507189    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.507516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.507489    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.507627 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.507613    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:52:14.509802 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.509773    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "kube-api-access-sxrmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.509953 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.509797    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.510020 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.509998    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:52:14.510072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.510041    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token" pod "d36e86b4-ae71-4466-82c8-6412967f2836" (UID: "d36e86b4-ae71-4466-82c8-6412967f2836"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.531713 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.531675    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:52:14.607591 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.607556    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.607909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.607865    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.608115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608098    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.608277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608261    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.608433 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608419    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.608595 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608580    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.608728 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608712    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.608874 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608841    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.608984 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.608971    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609099 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609074    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609146    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.609204 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609187    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609317    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.609386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609352    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609386 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609380    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\") pod \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\" (UID: \"ac2beee3-1fb9-4254-a409-357a3bfe0daf\") "
Nov 02 21:52:14.609487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609424    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609472    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.609487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609473    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.609645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.609515    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\") pod \"3c71dad7-7040-40d6-8f68-c83b010a8c95\" (UID: \"3c71dad7-7040-40d6-8f68-c83b010a8c95\") "
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610010    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610127    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610151    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrmk\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-kube-api-access-sxrmk\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610166    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610179    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610191    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610205    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610220    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610234    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36e86b4-ae71-4466-82c8-6412967f2836-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610247    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36e86b4-ae71-4466-82c8-6412967f2836-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610252 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610260    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36e86b4-ae71-4466-82c8-6412967f2836-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610273    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36e86b4-ae71-4466-82c8-6412967f2836-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.610951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610499    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.610951 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.610789    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:52:14.611389 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.611354    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.612002 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.611982    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.612368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.612333    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.612665 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.612645    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:52:14.612914 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.612892    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.613335 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.613306    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.613529 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.613512    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.614588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.614541    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.616114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.616091    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:52:14.617279 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.617253    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:52:14.619172 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.619147    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:52:14.619393 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.619376    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "kube-api-access-jqthj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.622559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.622524    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token" pod "3c71dad7-7040-40d6-8f68-c83b010a8c95" (UID: "3c71dad7-7040-40d6-8f68-c83b010a8c95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.629340 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.629311    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf" pod "ac2beee3-1fb9-4254-a409-357a3bfe0daf" (UID: "ac2beee3-1fb9-4254-a409-357a3bfe0daf"). InnerVolumeSpecName "kube-api-access-fltmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:52:14.649102 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.649070    4888 generic.go:356] "Generic (PLEG): container finished" podID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerID="4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236" exitCode=0
Nov 02 21:52:14.649309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.649128    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"
Nov 02 21:52:14.649309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.649168    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" event={"ID":"3c71dad7-7040-40d6-8f68-c83b010a8c95","Type":"ContainerDied","Data":"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"}
Nov 02 21:52:14.649309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.649247    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q" event={"ID":"3c71dad7-7040-40d6-8f68-c83b010a8c95","Type":"ContainerDied","Data":"51d8a3a71ae2225f13f0ca7f8b98fc20bfbe38138b27241e19dd37c842d35133"}
Nov 02 21:52:14.649309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.649295    4888 scope.go:122] "RemoveContainer" containerID="4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.657246    4888 generic.go:356] "Generic (PLEG): container finished" podID="d36e86b4-ae71-4466-82c8-6412967f2836" containerID="97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f" exitCode=0
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.657304    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.657334    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerDied","Data":"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"}
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.657398    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz" event={"ID":"d36e86b4-ae71-4466-82c8-6412967f2836","Type":"ContainerDied","Data":"13183a2ad3b3f54915a9b966fba822353b5d62e619dc78d369adc4503d8eb8c0"}
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.659261    4888 generic.go:356] "Generic (PLEG): container finished" podID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerID="0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c" exitCode=0
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.659323    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" event={"ID":"ac2beee3-1fb9-4254-a409-357a3bfe0daf","Type":"ContainerDied","Data":"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"}
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.659380    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s" event={"ID":"ac2beee3-1fb9-4254-a409-357a3bfe0daf","Type":"ContainerDied","Data":"2e2cbf90accd0e00a1740aab918d6a4fa0d081b8d6033a4f77bcbb27432c2e8e"}
Nov 02 21:52:14.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.659454    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"
Nov 02 21:52:14.663183 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.663163    4888 scope.go:122] "RemoveContainer" containerID="4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"
Nov 02 21:52:14.663497 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:14.663476    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236\": container with ID starting with 4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236 not found: ID does not exist" containerID="4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"
Nov 02 21:52:14.663589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.663506    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236"} err="failed to get container status \"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236\": rpc error: code = NotFound desc = could not find container \"4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236\": container with ID starting with 4db9dab4d5e4ae8fb24349224ec25e371ed48b5bb82a2865b6f06a587a526236 not found: ID does not exist"
Nov 02 21:52:14.663589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.663528    4888 scope.go:122] "RemoveContainer" containerID="97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"
Nov 02 21:52:14.677892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.677866    4888 scope.go:122] "RemoveContainer" containerID="f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"
Nov 02 21:52:14.695122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.695092    4888 scope.go:122] "RemoveContainer" containerID="97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"
Nov 02 21:52:14.696783 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:14.696758    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f\": container with ID starting with 97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f not found: ID does not exist" containerID="97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"
Nov 02 21:52:14.696999 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.696971    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f"} err="failed to get container status \"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f\": rpc error: code = NotFound desc = could not find container \"97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f\": container with ID starting with 97a9b39a54717aa14cc1a4e7a2b6e5acacf8d76643ed93fcb7517b1ef081e55f not found: ID does not exist"
Nov 02 21:52:14.697092 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.697082    4888 scope.go:122] "RemoveContainer" containerID="f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"
Nov 02 21:52:14.697494 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:14.697474    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7\": container with ID starting with f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7 not found: ID does not exist" containerID="f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"
Nov 02 21:52:14.697632 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.697613    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7"} err="failed to get container status \"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7\": rpc error: code = NotFound desc = could not find container \"f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7\": container with ID starting with f88314615fcacf6be9e6e54e68f3ea11d8fc11487301f87929f90d9799d6d6b7 not found: ID does not exist"
Nov 02 21:52:14.697730 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.697719    4888 scope.go:122] "RemoveContainer" containerID="0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.711911    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.711980    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.711996    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712010    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712026    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712041    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712080    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712094    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712108    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712136    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c71dad7-7040-40d6-8f68-c83b010a8c95-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712156    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqthj\" (UniqueName: \"kubernetes.io/projected/3c71dad7-7040-40d6-8f68-c83b010a8c95-kube-api-access-jqthj\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712174    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c71dad7-7040-40d6-8f68-c83b010a8c95-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712188    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fltmf\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-kube-api-access-fltmf\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712206    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712219    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.715891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.712232    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2beee3-1fb9-4254-a409-357a3bfe0daf-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:52:14.721358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.721327    4888 scope.go:122] "RemoveContainer" containerID="0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"
Nov 02 21:52:14.721766 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:14.721743    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c\": container with ID starting with 0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c not found: ID does not exist" containerID="0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"
Nov 02 21:52:14.721891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.721780    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c"} err="failed to get container status \"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c\": rpc error: code = NotFound desc = could not find container \"0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c\": container with ID starting with 0cef19fb85d9c662723e6f24d72f25d5cb9723c125b7f08c592e7b3e9041cd6c not found: ID does not exist"
Nov 02 21:52:14.912337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.912305    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:52:14.917155 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:14.917117    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s"]
Nov 02 21:52:15.020442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:15.020398    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:52:15.024357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:15.024295    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz"]
Nov 02 21:52:15.129981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:15.129933    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:52:15.132991 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:15.132963    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q"]
Nov 02 21:52:16.603120 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:16.603090    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" path="/var/lib/kubelet/pods/3c71dad7-7040-40d6-8f68-c83b010a8c95/volumes"
Nov 02 21:52:16.603625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:16.603612    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" path="/var/lib/kubelet/pods/ac2beee3-1fb9-4254-a409-357a3bfe0daf/volumes"
Nov 02 21:52:16.604125 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:16.604108    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" path="/var/lib/kubelet/pods/d36e86b4-ae71-4466-82c8-6412967f2836/volumes"
Nov 02 21:52:21.983074 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983034    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 21:52:21.983585 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983571    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983585    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983609    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983614    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983627    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy"
Nov 02 21:52:21.983722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983632    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy"
Nov 02 21:52:21.983722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983659    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy"
Nov 02 21:52:21.983722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983664    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy"
Nov 02 21:52:21.983842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983737    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c71dad7-7040-40d6-8f68-c83b010a8c95" containerName="istio-proxy"
Nov 02 21:52:21.983842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983746    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983755    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d36e86b4-ae71-4466-82c8-6412967f2836" containerName="istio-proxy"
Nov 02 21:52:21.983842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.983761    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac2beee3-1fb9-4254-a409-357a3bfe0daf" containerName="istio-proxy"
Nov 02 21:52:21.989040 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:21.989013    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.039349 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.039310    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 21:52:22.090196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.090157    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-11537e\""
Nov 02 21:52:22.110003 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.092048    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-11537e-dockercfg-jqvx5\""
Nov 02 21:52:22.124972 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.101877    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zshts\" (UniqueName: \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.125562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.125529    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.158442 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.158403    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 21:52:22.163700 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.163651    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.202612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.202569    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 21:52:22.234932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.232950    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.234932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.233050    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dkt7\" (UniqueName: \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.234932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.233216    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.234932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.233406    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zshts\" (UniqueName: \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.234932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.233792    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.264792 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:22.264753    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:52:22.287589 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.287555    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zshts\" (UniqueName: \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\") pod \"payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.303612 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.303567    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.335005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.334963    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dkt7\" (UniqueName: \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.335164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.335062    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.336159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.336134    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.389904 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.389529    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dkt7\" (UniqueName: \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\") pod \"payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.513210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.513119    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:22.677156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.677122    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 21:52:22.678112 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:52:22.678083    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b752e56_1a34_417a_9847_e99570238998.slice/crio-5351d384771af6693746abf71c13374ac5562ac2feef240650389504f5784795 WatchSource:0}: Error finding container 5351d384771af6693746abf71c13374ac5562ac2feef240650389504f5784795: Status 404 returned error can't find the container with id 5351d384771af6693746abf71c13374ac5562ac2feef240650389504f5784795
Nov 02 21:52:22.807170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.807062    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" event={"ID":"3b752e56-1a34-417a-9847-e99570238998","Type":"ContainerStarted","Data":"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"}
Nov 02 21:52:22.807170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.807129    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" event={"ID":"3b752e56-1a34-417a-9847-e99570238998","Type":"ContainerStarted","Data":"5351d384771af6693746abf71c13374ac5562ac2feef240650389504f5784795"}
Nov 02 21:52:22.807170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.807157    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:22.889895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.885819    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 21:52:22.917319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:22.917260    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" podStartSLOduration=1.917247637 podStartE2EDuration="1.917247637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:52:22.914367222 +0000 UTC m=+2372.458159136" watchObservedRunningTime="2026-11-02 21:52:22.917247637 +0000 UTC m=+2372.461039552"
Nov 02 21:52:23.814578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:23.814539    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" event={"ID":"ff96a539-9533-4745-b9ac-6c0c58a0f49f","Type":"ContainerStarted","Data":"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"}
Nov 02 21:52:23.815115 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:23.814599    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" event={"ID":"ff96a539-9533-4745-b9ac-6c0c58a0f49f","Type":"ContainerStarted","Data":"50017b01a062ffc7051024a7486ced21093445870ced7e8342640773e48879bd"}
Nov 02 21:52:23.925565 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:23.925511    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" podStartSLOduration=1.9254971570000001 podStartE2EDuration="1.925497157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:52:23.922344695 +0000 UTC m=+2373.466136609" watchObservedRunningTime="2026-11-02 21:52:23.925497157 +0000 UTC m=+2373.469289071"
Nov 02 21:52:24.821463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:24.821432    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:33.832164 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:33.832068    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 21:52:35.841820 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:35.841761    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 21:52:41.225012 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.224971    4888 generic.go:356] "Generic (PLEG): container finished" podID="851f0db2-7ab7-477a-95e9-310d2516314a" containerID="e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9" exitCode=0
Nov 02 21:52:41.225487 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.225041    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerDied","Data":"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"}
Nov 02 21:52:41.225534 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.225486    4888 scope.go:122] "RemoveContainer" containerID="e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"
Nov 02 21:52:41.227182 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.227151    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:41.227308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.227241    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:41.227308 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:41.227288    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:52:42.241170 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:42.241136    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerStarted","Data":"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"}
Nov 02 21:52:42.464138 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:42.464103    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:52:42.464344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:42.464323    4888 patch_prober.go:29] interesting pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Nov 02 21:52:42.464399 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:42.464379    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Nov 02 21:52:43.463770 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:43.463731    4888 patch_prober.go:29] interesting pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Nov 02 21:52:43.464319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:43.463796    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Nov 02 21:52:44.464638 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:44.464595    4888 patch_prober.go:29] interesting pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Nov 02 21:52:44.465230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:44.464660    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Nov 02 21:52:45.464226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:45.464189    4888 patch_prober.go:29] interesting pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Nov 02 21:52:45.464402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:45.464246    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Nov 02 21:52:46.464347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:46.464309    4888 patch_prober.go:29] interesting pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Nov 02 21:52:46.464837 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:46.464370    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Nov 02 21:52:47.566443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:47.566406    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:52:47.567309 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:47.567280    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:52:47.568357 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:47.568335    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 21:52:50.943005 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:50.942966    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:52:50.951517 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:52:50.951490    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:52:52.261944 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:52:52.261903    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:53:22.261108 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:53:22.261069    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:53:26.257348 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.257306    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 21:53:26.262570 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.262540    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.312625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.312583    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bcb857-openshift-default-dockercfg-5pzjb\""
Nov 02 21:53:26.359048 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359003    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359055    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359096    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359133    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359227    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359264    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359338    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359439    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.359649 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.359493    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hvm\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.365062 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.365013    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 21:53:26.460596 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.460551    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.460876 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.460828    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.460894    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.460931    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461063 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461028    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461068    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461247 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461143    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461256    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461272    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461358 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461332    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hvm\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.461616 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.461534    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.462177 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.462144    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.462746 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.462191    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.462927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.462793    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.465712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.465683    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.467887 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.466391    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.516471 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.516393    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hvm\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.516615 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.516525    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\") pod \"e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") " pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:26.578701 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:26.578664    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:27.079225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.079196    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 21:53:27.080497 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:53:27.080466    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f964ac_042b_4dfc_b166_2776d59f3673.slice/crio-36cfc93f7315951a0040981be2eda01664d1a7ded1872cc51c3df41130d0465c WatchSource:0}: Error finding container 36cfc93f7315951a0040981be2eda01664d1a7ded1872cc51c3df41130d0465c: Status 404 returned error can't find the container with id 36cfc93f7315951a0040981be2eda01664d1a7ded1872cc51c3df41130d0465c
Nov 02 21:53:27.083600 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.083569    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:27.083681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.083640    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:27.083681 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.083669    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:27.576869 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.576821    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerStarted","Data":"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"}
Nov 02 21:53:27.577526 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.576891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerStarted","Data":"36cfc93f7315951a0040981be2eda01664d1a7ded1872cc51c3df41130d0465c"}
Nov 02 21:53:27.579215 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.579193    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:27.688197 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:27.688143    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podStartSLOduration=1.688127857 podStartE2EDuration="1.688127857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:53:27.68702061 +0000 UTC m=+2437.230812528" watchObservedRunningTime="2026-11-02 21:53:27.688127857 +0000 UTC m=+2437.231919773"
Nov 02 21:53:28.586512 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:28.586474    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:29.590535 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:29.590500    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:29.591662 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:29.591641    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:53:34.123757 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.123664    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 21:53:34.132775 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.132750    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.153561 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.153515    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5z9\" (UniqueName: \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.156893 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.153790    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.166559 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.166518    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 21:53:34.208293 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.208235    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-bcb857-dockercfg-zxnd6\""
Nov 02 21:53:34.208293 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.208274    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-bcb857\""
Nov 02 21:53:34.255788 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.255745    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.256018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.255914    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5z9\" (UniqueName: \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.256625 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.256601    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.310263 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.310221    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5z9\" (UniqueName: \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\") pod \"payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.480377 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.480340    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:34.852538 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:34.852506    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 21:53:34.853941 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:53:34.853912    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2839fbe0_7269_435f_90cf_23f5eef50384.slice/crio-6c7b0bd159b8ef78185447e2a9590b97e12a4233564fa5a43b0b97d1b320c189 WatchSource:0}: Error finding container 6c7b0bd159b8ef78185447e2a9590b97e12a4233564fa5a43b0b97d1b320c189: Status 404 returned error can't find the container with id 6c7b0bd159b8ef78185447e2a9590b97e12a4233564fa5a43b0b97d1b320c189
Nov 02 21:53:35.638867 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:35.638805    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" event={"ID":"2839fbe0-7269-435f-90cf-23f5eef50384","Type":"ContainerStarted","Data":"2aecf43af3e294bbcd35ea9aa94db74cfb8bf6a833189a08f59f294332d398f0"}
Nov 02 21:53:35.639314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:35.638920    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:35.639314 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:35.638942    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" event={"ID":"2839fbe0-7269-435f-90cf-23f5eef50384","Type":"ContainerStarted","Data":"6c7b0bd159b8ef78185447e2a9590b97e12a4233564fa5a43b0b97d1b320c189"}
Nov 02 21:53:42.902187 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:42.902122    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" podStartSLOduration=8.902096758999999 podStartE2EDuration="8.902096759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:53:35.745975837 +0000 UTC m=+2445.289767752" watchObservedRunningTime="2026-11-02 21:53:42.902096759 +0000 UTC m=+2452.445888677"
Nov 02 21:53:42.903192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:42.902550    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:53:46.649511 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:46.649481    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 21:53:51.202599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:51.202183    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:53:51.202599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:51.202257    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:53:52.264281 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:53:52.264239    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:53:52.795121 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.795079    4888 generic.go:356] "Generic (PLEG): container finished" podID="40bfd517-4735-4472-8a0a-963f57391648" containerID="729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a" exitCode=0
Nov 02 21:53:52.795306 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.795153    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerDied","Data":"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"}
Nov 02 21:53:52.795644 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.795629    4888 scope.go:122] "RemoveContainer" containerID="729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"
Nov 02 21:53:52.797505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.797476    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:52.797630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.797554    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:52.797630 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:52.797609    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:53:53.804243 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:53.804211    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerStarted","Data":"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"}
Nov 02 21:53:54.504235 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:54.504203    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:53:54.504482 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:54.504459    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 21:53:54.504542 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:54.504521    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 21:53:55.504456 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:55.504421    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 21:53:55.504901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:55.504479    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 21:53:56.506716 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:56.504009    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 21:53:56.506716 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:56.504073    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 21:53:57.504558 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:57.504523    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 21:53:57.504741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:57.504588    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 21:53:57.950696 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:57.950631    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main" containerID="cri-o://3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15" gracePeriod=45
Nov 02 21:53:58.272146 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.272111    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:53:58.351530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351493    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.351737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351595    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\" (UniqueName: \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.351737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351639    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.351737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351668    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.351950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351764    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.351950 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.351795    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\") pod \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\" (UID: \"9f32883b-8fc9-404a-8c2c-774c83ca5d70\") "
Nov 02 21:53:58.352574 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.352548    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:53:58.352774 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.352636    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:53:58.353377 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.352806    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:53:58.356516 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.356493    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:53:58.356652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.356525    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "kube-api-access-c9wpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:53:58.357114 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.357092    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm" pod "9f32883b-8fc9-404a-8c2c-774c83ca5d70" (UID: "9f32883b-8fc9-404a-8c2c-774c83ca5d70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:53:58.453505 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.453468    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.453712 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.453699    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9wpn\" (UniqueName: \"kubernetes.io/projected/9f32883b-8fc9-404a-8c2c-774c83ca5d70-kube-api-access-c9wpn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.453778 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.453770    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.453840 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.453832    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.453933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.453923    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.454009 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.454002    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32883b-8fc9-404a-8c2c-774c83ca5d70-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:53:58.504790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.504378    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 21:53:58.504790 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.504427    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 21:53:58.860653 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.860575    4888 generic.go:356] "Generic (PLEG): container finished" podID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerID="3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15" exitCode=0
Nov 02 21:53:58.860883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.860621    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"
Nov 02 21:53:58.860967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.860639    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" event={"ID":"9f32883b-8fc9-404a-8c2c-774c83ca5d70","Type":"ContainerDied","Data":"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"}
Nov 02 21:53:58.861029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.860996    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k" event={"ID":"9f32883b-8fc9-404a-8c2c-774c83ca5d70","Type":"ContainerDied","Data":"b233521315588309d5b118cf2275a5d659e5f957a91bd703953282c546e10b83"}
Nov 02 21:53:58.861029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.861025    4888 scope.go:122] "RemoveContainer" containerID="3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"
Nov 02 21:53:58.873993 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.873968    4888 scope.go:122] "RemoveContainer" containerID="3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"
Nov 02 21:53:58.874290 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:53:58.874270    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15\": container with ID starting with 3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15 not found: ID does not exist" containerID="3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"
Nov 02 21:53:58.874353 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:58.874304    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15"} err="failed to get container status \"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15\": rpc error: code = NotFound desc = could not find container \"3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15\": container with ID starting with 3bb714e58d9f993a578553fbbd2a848b4df399504c40b4acb734981f88d2be15 not found: ID does not exist"
Nov 02 21:53:59.040544 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:59.040507    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:53:59.043571 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:59.043544    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-ef6b43/premium-w5-ef6b43-kserve-7d66dcfcdc-pcr6k"]
Nov 02 21:53:59.510656 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:59.510572    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:53:59.511136 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:59.511118    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:53:59.512081 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:53:59.512055    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 21:54:00.602111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:00.602077    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" path="/var/lib/kubelet/pods/9f32883b-8fc9-404a-8c2c-774c83ca5d70/volumes"
Nov 02 21:54:21.066304 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.066262    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:54:21.067051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.066990    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main"
Nov 02 21:54:21.067051 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.067011    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main"
Nov 02 21:54:21.067231 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.067216    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f32883b-8fc9-404a-8c2c-774c83ca5d70" containerName="main"
Nov 02 21:54:21.073141 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.073118    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.120795 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.120747    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:54:21.164402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.164366    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d4f799\"/\"default-dockercfg-6c9fn\""
Nov 02 21:54:21.182163 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.182128    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d4f799\"/\"test-model-d4f799-kserve-self-signed-certs\""
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txs62\" (UniqueName: \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226308    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226358    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226437    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226480    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.226755 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.226521    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.238917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.201931    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d4f799\"/\"openshift-service-ca.crt\""
Nov 02 21:54:21.239336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.239311    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d4f799\"/\"kube-root-ca.crt\""
Nov 02 21:54:21.345965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.340171    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.345965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.340248    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.345965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.340382    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txs62\" (UniqueName: \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.345965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.340454    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.345965 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.340525    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.365946 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.365907    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.366652 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.366623    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.369043 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.368001    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.369219 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.368977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.419873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.412014    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txs62\" (UniqueName: \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.421123 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.421091    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.430214 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.430143    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\") pod \"test-model-d4f799-kserve-855cd6ff49-2fl5t\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") " pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:21.706872 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:21.703099    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:22.614911 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:22.613564    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:22.702878 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:22.697155    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:22.706952 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:54:22.706572    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:54:22.798931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:22.797741    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:54:23.098599 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:23.098489    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" event={"ID":"67375468-ed62-4a72-9704-ecbb8c507ed0","Type":"ContainerStarted","Data":"013928c88fed791f15645bc0b24ace8b9c108c3291819430410970db485be30b"}
Nov 02 21:54:24.106443 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:24.106409    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" event={"ID":"67375468-ed62-4a72-9704-ecbb8c507ed0","Type":"ContainerStarted","Data":"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"}
Nov 02 21:54:24.256174 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:24.256117    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" podStartSLOduration=3.256098835 podStartE2EDuration="3.256098835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:54:24.255611371 +0000 UTC m=+2493.799403289" watchObservedRunningTime="2026-11-02 21:54:24.256098835 +0000 UTC m=+2493.799890751"
Nov 02 21:54:24.703767 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:24.703732    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:31.704563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:31.704527    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:31.718556 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:31.718522    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:32.176387 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:54:32.176349    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:54:52.260696 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:54:52.260653    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:55:22.260822 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:55:22.260778    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:55:32.304185 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.304094    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 21:55:32.308742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.308715    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.360677 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.360640    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 21:55:32.382664 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.382618    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.382842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.382667    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.382994 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.382963    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsbhh\" (UniqueName: \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.383047 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.383013    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.383166 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.383130    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.383286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.383262    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.404586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.404548    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-686bbf\"/\"kube-root-ca.crt\""
Nov 02 21:55:32.404586 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.404577    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-686bbf\"/\"openshift-service-ca.crt\""
Nov 02 21:55:32.406230 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.406211    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-686bbf\"/\"default-dockercfg-4kjvz\""
Nov 02 21:55:32.406286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.406256    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-686bbf\"/\"test-model-686bbf-kserve-self-signed-certs\""
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485002    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsbhh\" (UniqueName: \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485119    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485162    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485273    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485348    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.487830 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.485371    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.492425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.491464    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.492745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.492163    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.492926 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.486876    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.496757 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.496729    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.497013 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.496993    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.545540 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.545507    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsbhh\" (UniqueName: \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\") pod \"test-model-686bbf-kserve-757d9f9dbc-8wkqc\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") " pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:32.621718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:32.621620    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:33.002891 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:33.002842    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 21:55:33.004607 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:55:33.004580    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36ce2bd1_c012_44f1_85f9_9311fce8249c.slice/crio-54a8e881cb548df61b4e9ec4606cac2b99ff4e6959eb45b7a489a45011cdde50 WatchSource:0}: Error finding container 54a8e881cb548df61b4e9ec4606cac2b99ff4e6959eb45b7a489a45011cdde50: Status 404 returned error can't find the container with id 54a8e881cb548df61b4e9ec4606cac2b99ff4e6959eb45b7a489a45011cdde50
Nov 02 21:55:33.604577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:33.604539    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" event={"ID":"36ce2bd1-c012-44f1-85f9-9311fce8249c","Type":"ContainerStarted","Data":"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"}
Nov 02 21:55:33.604577 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:33.604586    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" event={"ID":"36ce2bd1-c012-44f1-85f9-9311fce8249c","Type":"ContainerStarted","Data":"54a8e881cb548df61b4e9ec4606cac2b99ff4e6959eb45b7a489a45011cdde50"}
Nov 02 21:55:33.622421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:33.622389    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:33.715104 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:33.715049    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" podStartSLOduration=1.7150347689999998 podStartE2EDuration="1.715034769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:33.712408014 +0000 UTC m=+2563.256199930" watchObservedRunningTime="2026-11-02 21:55:33.715034769 +0000 UTC m=+2563.258826684"
Nov 02 21:55:42.621913 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:42.621875    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:42.634909 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:42.634882    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:42.715264 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:42.715236    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 21:55:52.260672 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:55:52.260627    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:55:55.490764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:55.490728    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:55:55.491305 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:55.491006    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" podUID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" containerName="maas-api" containerID="cri-o://ca4f0e7ff381e87bf36b155c1267b6445c90ccd3f8cd8acb7770857af92bb3b6" gracePeriod=30
Nov 02 21:55:55.817917 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:55.817810    4888 generic.go:356] "Generic (PLEG): container finished" podID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" containerID="ca4f0e7ff381e87bf36b155c1267b6445c90ccd3f8cd8acb7770857af92bb3b6" exitCode=0
Nov 02 21:55:55.818126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:55.818101    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" event={"ID":"990bdd1c-ddd7-4025-a57c-8dc0aa083950","Type":"ContainerDied","Data":"ca4f0e7ff381e87bf36b155c1267b6445c90ccd3f8cd8acb7770857af92bb3b6"}
Nov 02 21:55:55.907129 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:55.907093    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:55:56.039481 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.039441    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\") pod \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") "
Nov 02 21:55:56.039692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.039509    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\" (UniqueName: \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\") pod \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") "
Nov 02 21:55:56.039764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.039696    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") pod \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") "
Nov 02 21:55:56.039764 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.039737    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\") pod \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\" (UID: \"990bdd1c-ddd7-4025-a57c-8dc0aa083950\") "
Nov 02 21:55:56.040645 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.040615    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca" pod "990bdd1c-ddd7-4025-a57c-8dc0aa083950" (UID: "990bdd1c-ddd7-4025-a57c-8dc0aa083950"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:55:56.042918 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.042799    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls" pod "990bdd1c-ddd7-4025-a57c-8dc0aa083950" (UID: "990bdd1c-ddd7-4025-a57c-8dc0aa083950"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:55:56.045082 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.045052    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs" pod "990bdd1c-ddd7-4025-a57c-8dc0aa083950" (UID: "990bdd1c-ddd7-4025-a57c-8dc0aa083950"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:55:56.046376 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.046348    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp" pod "990bdd1c-ddd7-4025-a57c-8dc0aa083950" (UID: "990bdd1c-ddd7-4025-a57c-8dc0aa083950"). InnerVolumeSpecName "kube-api-access-665lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:55:56.143328 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.142332    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:55:56.143328 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.142369    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990bdd1c-ddd7-4025-a57c-8dc0aa083950-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:55:56.143328 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.142383    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990bdd1c-ddd7-4025-a57c-8dc0aa083950-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:55:56.143328 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.142400    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-665lp\" (UniqueName: \"kubernetes.io/projected/990bdd1c-ddd7-4025-a57c-8dc0aa083950-kube-api-access-665lp\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:55:56.827337 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.827302    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"
Nov 02 21:55:56.827842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.827340    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9" event={"ID":"990bdd1c-ddd7-4025-a57c-8dc0aa083950","Type":"ContainerDied","Data":"f7312b1455e87fd42a7c8a9a100bf4c2fb6773f1ef9d2a6106c2bcc4d2142f6e"}
Nov 02 21:55:56.827842 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.827411    4888 scope.go:122] "RemoveContainer" containerID="ca4f0e7ff381e87bf36b155c1267b6445c90ccd3f8cd8acb7770857af92bb3b6"
Nov 02 21:55:56.983924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:56.983874    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:55:57.031320 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:57.031282    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ef6b43-6665f968f8-6lhh9"]
Nov 02 21:55:58.600563 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:55:58.600524    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" path="/var/lib/kubelet/pods/990bdd1c-ddd7-4025-a57c-8dc0aa083950/volumes"
Nov 02 21:56:04.340126 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:04.340081    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:56:04.341029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:04.340997    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy" containerID="cri-o://a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc" gracePeriod=30
Nov 02 21:56:09.610718 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.610644    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:56:09.630402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630372    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630456    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630496    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630528    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630607 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630567    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630607    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630637    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630748    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.630927 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.630873    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\") pod \"f700027b-6535-4875-bab0-270c68fff872\" (UID: \"f700027b-6535-4875-bab0-270c68fff872\") "
Nov 02 21:56:09.631142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631031    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:09.631373 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631338    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:56:09.631593 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631575    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.631685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631599    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f700027b-6535-4875-bab0-270c68fff872-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.631685 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631602    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:09.631824 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.631805    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:09.632369 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.632342    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:09.633116 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.633024    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:09.633116 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.633098    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:56:09.633260 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.633224    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:09.634034 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.634011    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b" pod "f700027b-6535-4875-bab0-270c68fff872" (UID: "f700027b-6535-4875-bab0-270c68fff872"). InnerVolumeSpecName "kube-api-access-hqn7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:09.732529 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732491    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732529 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732524    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732529 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732534    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732543    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732552    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f700027b-6535-4875-bab0-270c68fff872-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732560    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f700027b-6535-4875-bab0-270c68fff872-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.732817 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.732568    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqn7b\" (UniqueName: \"kubernetes.io/projected/f700027b-6535-4875-bab0-270c68fff872-kube-api-access-hqn7b\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:56:09.914244 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.914212    4888 generic.go:356] "Generic (PLEG): container finished" podID="f700027b-6535-4875-bab0-270c68fff872" containerID="a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc" exitCode=0
Nov 02 21:56:09.914421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.914258    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" event={"ID":"f700027b-6535-4875-bab0-270c68fff872","Type":"ContainerDied","Data":"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"}
Nov 02 21:56:09.914421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.914297    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk" event={"ID":"f700027b-6535-4875-bab0-270c68fff872","Type":"ContainerDied","Data":"863f134623430f50f96c8257a008c5e4335b518522c3a6b5cea88958938fc821"}
Nov 02 21:56:09.914421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.914319    4888 scope.go:122] "RemoveContainer" containerID="a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"
Nov 02 21:56:09.914421 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.914259    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"
Nov 02 21:56:09.927901 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.927837    4888 scope.go:122] "RemoveContainer" containerID="a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"
Nov 02 21:56:09.928286 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:56:09.928268    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc\": container with ID starting with a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc not found: ID does not exist" containerID="a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"
Nov 02 21:56:09.928336 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:09.928297    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc"} err="failed to get container status \"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc\": rpc error: code = NotFound desc = could not find container \"a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc\": container with ID starting with a4380070a4b7eecfd6717737a8b15f8a78f39113c4209b6d99124ae5ac4959dc not found: ID does not exist"
Nov 02 21:56:10.072752 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:10.072718    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:56:10.078278 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:10.078245    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk"]
Nov 02 21:56:10.602029 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:10.601995    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f700027b-6535-4875-bab0-270c68fff872" path="/var/lib/kubelet/pods/f700027b-6535-4875-bab0-270c68fff872/volumes"
Nov 02 21:56:22.262548 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:56:22.262496    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:56:24.916094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916058    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 21:56:24.916687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916614    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy"
Nov 02 21:56:24.916687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916625    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy"
Nov 02 21:56:24.916687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916666    4888 cpu_manager.go:414] "containerMap: removing container" podUID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" containerName="maas-api"
Nov 02 21:56:24.916687 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916673    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" containerName="maas-api"
Nov 02 21:56:24.916938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916768    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="990bdd1c-ddd7-4025-a57c-8dc0aa083950" containerName="maas-api"
Nov 02 21:56:24.916938 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.916778    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f700027b-6535-4875-bab0-270c68fff872" containerName="istio-proxy"
Nov 02 21:56:24.920347 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.920325    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.952562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952519    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.952814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952646    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.952814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952711    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.952814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952729    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.952814 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952793    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqqwf\" (UniqueName: \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.953084 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.952908    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:24.972742 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.972708    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 21:56:24.997234 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:24.997195    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c1243c\"/\"test-model-c1243c-kserve-self-signed-certs\""
Nov 02 21:56:25.016737 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.016695    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c1243c\"/\"default-dockercfg-2nkqh\""
Nov 02 21:56:25.016967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.016760    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c1243c\"/\"kube-root-ca.crt\""
Nov 02 21:56:25.016967 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.016829    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c1243c\"/\"openshift-service-ca.crt\""
Nov 02 21:56:25.032578 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.032529    4888 generic.go:356] "Generic (PLEG): container finished" podID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerID="7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d" exitCode=0
Nov 02 21:56:25.032753 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.032603    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerDied","Data":"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"}
Nov 02 21:56:25.033076 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.033061    4888 scope.go:122] "RemoveContainer" containerID="7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"
Nov 02 21:56:25.034934 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.034903    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:56:25.035026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.034974    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:56:25.035026 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.035008    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 21:56:25.054079 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054054    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054120    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054185    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054209    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054247    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqqwf\" (UniqueName: \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054274    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054528 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054443    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054721 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054698    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.054795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.056539 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.056517    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.057168 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.057150    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.108217 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.108192    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqqwf\" (UniqueName: \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\") pod \"test-model-c1243c-kserve-67f78f8959-dlw5m\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") " pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.233462 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.233366    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:25.629554 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:25.629522    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 21:56:25.631694 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:56:25.631661    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a5476b7_97d8_48be_99de_37fd520e0f8c.slice/crio-d2dcea1a1d995941a199c31ca82d7fe092d21958ce6f9c327c59afb15a23734f WatchSource:0}: Error finding container d2dcea1a1d995941a199c31ca82d7fe092d21958ce6f9c327c59afb15a23734f: Status 404 returned error can't find the container with id d2dcea1a1d995941a199c31ca82d7fe092d21958ce6f9c327c59afb15a23734f
Nov 02 21:56:26.045463 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.045413    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" event={"ID":"1a5476b7-97d8-48be-99de-37fd520e0f8c","Type":"ContainerStarted","Data":"823a6d69fed7015940e72e7065fcdd8f6239a0c3b46b037f7732e026e884b8a6"}
Nov 02 21:56:26.045998 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.045481    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" event={"ID":"1a5476b7-97d8-48be-99de-37fd520e0f8c","Type":"ContainerStarted","Data":"d2dcea1a1d995941a199c31ca82d7fe092d21958ce6f9c327c59afb15a23734f"}
Nov 02 21:56:26.048226 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.048194    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerStarted","Data":"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"}
Nov 02 21:56:26.155282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.155224    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" podStartSLOduration=2.155207733 podStartE2EDuration="2.155207733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:26.153191366 +0000 UTC m=+2615.696983282" watchObservedRunningTime="2026-11-02 21:56:26.155207733 +0000 UTC m=+2615.698999649"
Nov 02 21:56:26.235879 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.235000    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:26.579553 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.579510    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:56:26.580670 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.580645    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:26.580779 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:26.580695    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:27.579969 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:27.579928    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:27.580461 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:27.579993    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:28.579122 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:28.579085    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:28.579321 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:28.579142    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:29.039210 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.039155    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 21:56:29.046892 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.046220    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.097992 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.097957    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.138643    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.138718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfvx\" (UniqueName: \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.138768    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.138903    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.138955    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.139294 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.139004    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.142425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.142136    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-70d237\"/\"kube-root-ca.crt\""
Nov 02 21:56:29.142425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.142261    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-70d237\"/\"openshift-service-ca.crt\""
Nov 02 21:56:29.143947 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.143921    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-70d237\"/\"default-dockercfg-q624b\""
Nov 02 21:56:29.144097 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.143993    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-70d237\"/\"test-model-70d237-kserve-self-signed-certs\""
Nov 02 21:56:29.242932 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.242838    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243140 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.242954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243140 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243002    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243145    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243185    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfvx\" (UniqueName: \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243229    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243344 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243286    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243573 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243545    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.243651 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.243627    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.246157 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.246126    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.246697 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.246671    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.298111 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.298040    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfvx\" (UniqueName: \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\") pod \"test-model-70d237-kserve-6c47468c5c-gw6vf\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") " pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.360974 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.360926    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:29.625413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.599965    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:56:29.625413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.623519    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:29.625413 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:29.623575    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:30.003546 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:30.003513    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 21:56:30.003885 ip-10-0-130-83 kubenswrapper[4888]: W1102 21:56:30.003860    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea1b6e79_6644_41c6_ba68_f3d77bf356d3.slice/crio-d25625032b3c0e83a5e439d28d1a3eb7a107e2a195422f1347846f23faf2f190 WatchSource:0}: Error finding container d25625032b3c0e83a5e439d28d1a3eb7a107e2a195422f1347846f23faf2f190: Status 404 returned error can't find the container with id d25625032b3c0e83a5e439d28d1a3eb7a107e2a195422f1347846f23faf2f190
Nov 02 21:56:30.080152 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:30.080109    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" event={"ID":"ea1b6e79-6644-41c6-ba68-f3d77bf356d3","Type":"ContainerStarted","Data":"d25625032b3c0e83a5e439d28d1a3eb7a107e2a195422f1347846f23faf2f190"}
Nov 02 21:56:30.579646 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:30.579606    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:30.579889 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:30.579664    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:31.088190 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:31.088151    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" event={"ID":"ea1b6e79-6644-41c6-ba68-f3d77bf356d3","Type":"ContainerStarted","Data":"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"}
Nov 02 21:56:31.197588 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:31.197522    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" podStartSLOduration=3.197506394 podStartE2EDuration="3.197506394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:31.195689082 +0000 UTC m=+2620.739480998" watchObservedRunningTime="2026-11-02 21:56:31.197506394 +0000 UTC m=+2620.741298307"
Nov 02 21:56:31.369364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:31.369253    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:31.579303 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:31.579264    4888 patch_prober.go:29] interesting pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Nov 02 21:56:31.579504 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:31.579325    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Nov 02 21:56:32.584620 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:32.584585    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:56:32.585784 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:32.585763    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 21:56:35.234191 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:35.234149    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:35.247282 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:35.247251    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:36.138825 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:36.138787    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 21:56:39.362072 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:39.362032    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:39.374923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:39.374888    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:40.157057 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:56:40.157027    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 21:56:52.265545 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:56:52.265502    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:57:13.735635 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:13.735539    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:57:13.736184 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:13.735831    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy" containerID="cri-o://26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f" gracePeriod=30
Nov 02 21:57:19.026530 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.026501    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:57:19.210368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210328    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210396    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210422    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210465    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210523    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210665    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210725    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.210783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210742    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:57:19.210783 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210758    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.211036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210788    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:57:19.211036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210942    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:57:19.211036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.210969    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\") pod \"d63d53d7-46ef-4203-895b-2c7d4db0945b\" (UID: \"d63d53d7-46ef-4203-895b-2c7d4db0945b\") "
Nov 02 21:57:19.211330 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211307    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:57:19.211494 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211465    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:57:19.211736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211632    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.211736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211655    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.211736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211685    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.211736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211699    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63d53d7-46ef-4203-895b-2c7d4db0945b-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.211736 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.211713    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.213225 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.213203    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:57:19.213319 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.213280    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:57:19.213446 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.213421    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:57:19.213623 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.213601    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt" pod "d63d53d7-46ef-4203-895b-2c7d4db0945b" (UID: "d63d53d7-46ef-4203-895b-2c7d4db0945b"). InnerVolumeSpecName "kube-api-access-sbmnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:57:19.313109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.313021    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.313109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.313053    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.313109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.313063    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbmnt\" (UniqueName: \"kubernetes.io/projected/d63d53d7-46ef-4203-895b-2c7d4db0945b-kube-api-access-sbmnt\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.313109 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.313074    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63d53d7-46ef-4203-895b-2c7d4db0945b-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:19.474545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.474504    4888 generic.go:356] "Generic (PLEG): container finished" podID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerID="26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f" exitCode=0
Nov 02 21:57:19.474722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.474581    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"
Nov 02 21:57:19.474722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.474589    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerDied","Data":"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"}
Nov 02 21:57:19.474722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.474678    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk" event={"ID":"d63d53d7-46ef-4203-895b-2c7d4db0945b","Type":"ContainerDied","Data":"b6a3720023c7a9907b9127a23d2a0d406a36a4d0292ee0ad1403660b1c817b7a"}
Nov 02 21:57:19.474722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.474700    4888 scope.go:122] "RemoveContainer" containerID="26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"
Nov 02 21:57:19.486142 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.486120    4888 scope.go:122] "RemoveContainer" containerID="56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"
Nov 02 21:57:19.495545 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.495520    4888 scope.go:122] "RemoveContainer" containerID="26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"
Nov 02 21:57:19.495873 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:57:19.495834    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f\": container with ID starting with 26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f not found: ID does not exist" containerID="26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"
Nov 02 21:57:19.495963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.495886    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f"} err="failed to get container status \"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f\": rpc error: code = NotFound desc = could not find container \"26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f\": container with ID starting with 26c2bc6cfe07b31f8d0e80ba9b7e0037ea7291c34773c4f2447a761c04bc448f not found: ID does not exist"
Nov 02 21:57:19.495963 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.495905    4888 scope.go:122] "RemoveContainer" containerID="56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"
Nov 02 21:57:19.496176 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:57:19.496155    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3\": container with ID starting with 56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3 not found: ID does not exist" containerID="56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"
Nov 02 21:57:19.496221 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.496184    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3"} err="failed to get container status \"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3\": rpc error: code = NotFound desc = could not find container \"56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3\": container with ID starting with 56ed1bfeb8e3a8006894143d82ae4a327b4bea5fce5c361b9a96a20ef068b0a3 not found: ID does not exist"
Nov 02 21:57:19.631159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.631051    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:57:19.634457 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:19.634430    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk"]
Nov 02 21:57:20.602793 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:20.602754    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" path="/var/lib/kubelet/pods/d63d53d7-46ef-4203-895b-2c7d4db0945b/volumes"
Nov 02 21:57:22.303135 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:57:22.303085    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:57:25.937692 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:25.937655    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:57:30.938390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:30.938322    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" podUID="aff3bdee-9072-4a1e-b88b-86e20015c001" containerName="payload-processing" containerID="cri-o://c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f" gracePeriod=25
Nov 02 21:57:31.222719 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.222688    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:57:31.283224 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.283183    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\" (UniqueName: \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\") pod \"aff3bdee-9072-4a1e-b88b-86e20015c001\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") "
Nov 02 21:57:31.283396 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.283337    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\") pod \"aff3bdee-9072-4a1e-b88b-86e20015c001\" (UID: \"aff3bdee-9072-4a1e-b88b-86e20015c001\") "
Nov 02 21:57:31.283741 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.283717    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume" pod "aff3bdee-9072-4a1e-b88b-86e20015c001" (UID: "aff3bdee-9072-4a1e-b88b-86e20015c001"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:57:31.285425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.285407    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl" pod "aff3bdee-9072-4a1e-b88b-86e20015c001" (UID: "aff3bdee-9072-4a1e-b88b-86e20015c001"). InnerVolumeSpecName "kube-api-access-ll9wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:57:31.384906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.384869    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff3bdee-9072-4a1e-b88b-86e20015c001-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:31.384906 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.384901    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll9wl\" (UniqueName: \"kubernetes.io/projected/aff3bdee-9072-4a1e-b88b-86e20015c001-kube-api-access-ll9wl\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:57:31.559007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.558911    4888 generic.go:356] "Generic (PLEG): container finished" podID="aff3bdee-9072-4a1e-b88b-86e20015c001" containerID="c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f" exitCode=0
Nov 02 21:57:31.559007 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.558965    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"
Nov 02 21:57:31.559240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.558966    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" event={"ID":"aff3bdee-9072-4a1e-b88b-86e20015c001","Type":"ContainerDied","Data":"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"}
Nov 02 21:57:31.559240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.559073    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67" event={"ID":"aff3bdee-9072-4a1e-b88b-86e20015c001","Type":"ContainerDied","Data":"b6767d5334d323ce18fe2dc9ca9e99d261b70d04c8dd48ac165b3f3becbbedc7"}
Nov 02 21:57:31.559240 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.559100    4888 scope.go:122] "RemoveContainer" containerID="c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"
Nov 02 21:57:31.570018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.569999    4888 scope.go:122] "RemoveContainer" containerID="c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"
Nov 02 21:57:31.570359 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:57:31.570339    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f\": container with ID starting with c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f not found: ID does not exist" containerID="c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"
Nov 02 21:57:31.570409 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.570369    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f"} err="failed to get container status \"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f\": rpc error: code = NotFound desc = could not find container \"c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f\": container with ID starting with c79b1bc37a25f4dc0834248b7a78de4355be71a4d8afc683eb7bc04544a6012f not found: ID does not exist"
Nov 02 21:57:31.717748 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.717711    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:57:31.764566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:31.764527    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67"]
Nov 02 21:57:32.601402 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:32.601367    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aff3bdee-9072-4a1e-b88b-86e20015c001" path="/var/lib/kubelet/pods/aff3bdee-9072-4a1e-b88b-86e20015c001/volumes"
Nov 02 21:57:50.985587 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:50.985555    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:57:50.995175 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:57:50.995142    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 21:57:52.261446 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:57:52.261409    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:58:22.260606 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:58:22.260563    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:58:52.261515 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:58:52.261472    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:59:19.736403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:19.736362    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:59:22.263021 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:22.262980    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:59:31.672238 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:31.672203    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:59:31.773061 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:31.773028    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:59:34.758323 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:34.758279    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" podUID="67375468-ed62-4a72-9704-ecbb8c507ed0" containerName="main" containerID="cri-o://0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6" gracePeriod=45
Nov 02 21:59:35.043016 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.042988    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:59:35.136438 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136402    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\" (UniqueName: \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136461    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136512    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136563    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136642 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136636    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\" (UniqueName: \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136699    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\") pod \"67375468-ed62-4a72-9704-ecbb8c507ed0\" (UID: \"67375468-ed62-4a72-9704-ecbb8c507ed0\") "
Nov 02 21:59:35.136924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136840    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:35.136924 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136903    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:35.137087 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.136927    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:35.137286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.137266    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.137286 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.137286    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.137434 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.137295    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.138758 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.138729    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:35.138923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.138833    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:59:35.138923 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.138889    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62" pod "67375468-ed62-4a72-9704-ecbb8c507ed0" (UID: "67375468-ed62-4a72-9704-ecbb8c507ed0"). InnerVolumeSpecName "kube-api-access-txs62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:35.238562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.238526    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txs62\" (UniqueName: \"kubernetes.io/projected/67375468-ed62-4a72-9704-ecbb8c507ed0-kube-api-access-txs62\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.238562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.238561    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67375468-ed62-4a72-9704-ecbb8c507ed0-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.238562 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.238571    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67375468-ed62-4a72-9704-ecbb8c507ed0-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:35.414192 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.414155    4888 generic.go:356] "Generic (PLEG): container finished" podID="67375468-ed62-4a72-9704-ecbb8c507ed0" containerID="0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6" exitCode=0
Nov 02 21:59:35.414403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.414201    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"
Nov 02 21:59:35.414403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.414252    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" event={"ID":"67375468-ed62-4a72-9704-ecbb8c507ed0","Type":"ContainerDied","Data":"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"}
Nov 02 21:59:35.414403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.414298    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t" event={"ID":"67375468-ed62-4a72-9704-ecbb8c507ed0","Type":"ContainerDied","Data":"013928c88fed791f15645bc0b24ace8b9c108c3291819430410970db485be30b"}
Nov 02 21:59:35.414403 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.414319    4888 scope.go:122] "RemoveContainer" containerID="0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"
Nov 02 21:59:35.425196 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.425176    4888 scope.go:122] "RemoveContainer" containerID="0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"
Nov 02 21:59:35.425521 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:35.425492    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6\": container with ID starting with 0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6 not found: ID does not exist" containerID="0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"
Nov 02 21:59:35.425624 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.425527    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6"} err="failed to get container status \"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6\": rpc error: code = NotFound desc = could not find container \"0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6\": container with ID starting with 0f4fe3f9e06356c283995151f7b6714cf56aeb8a112db630b0161450bc6b65d6 not found: ID does not exist"
Nov 02 21:59:35.570572 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.570536    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:59:35.574322 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:35.574291    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d4f799/test-model-d4f799-kserve-855cd6ff49-2fl5t"]
Nov 02 21:59:36.602981 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:36.602947    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67375468-ed62-4a72-9704-ecbb8c507ed0" path="/var/lib/kubelet/pods/67375468-ed62-4a72-9704-ecbb8c507ed0/volumes"
Nov 02 21:59:36.672500 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:36.672459    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing" containerID="cri-o://f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a" gracePeriod=25
Nov 02 21:59:36.773531 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:36.773468    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing" containerID="cri-o://e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b" gracePeriod=25
Nov 02 21:59:38.253083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:38.253037    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 21:59:38.253083 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:38.253070    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 21:59:48.252895 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:48.252841    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 21:59:48.253380 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:48.252839    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 21:59:52.262983 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:52.262938    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 21:59:52.503931 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:52.503812    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:59:52.504156 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:52.504071    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" containerID="cri-o://a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36" gracePeriod=30
Nov 02 21:59:55.301659 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:55.301621    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 21:59:55.302200 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:55.301862    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" containerID="cri-o://a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea" gracePeriod=30
Nov 02 21:59:57.157886 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.157839    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:59:57.204949 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.204920    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:59:57.265883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.265018    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\") pod \"67265b28-0d37-439c-bf71-bf137fc4a470\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") "
Nov 02 21:59:57.265883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.265114    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\") pod \"2772247b-ee45-43aa-bfbe-ec4697942d72\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") "
Nov 02 21:59:57.265883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.265275    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\" (UniqueName: \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\") pod \"2772247b-ee45-43aa-bfbe-ec4697942d72\" (UID: \"2772247b-ee45-43aa-bfbe-ec4697942d72\") "
Nov 02 21:59:57.265883 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.265317    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\" (UniqueName: \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\") pod \"67265b28-0d37-439c-bf71-bf137fc4a470\" (UID: \"67265b28-0d37-439c-bf71-bf137fc4a470\") "
Nov 02 21:59:57.269390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.267725    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume" pod "67265b28-0d37-439c-bf71-bf137fc4a470" (UID: "67265b28-0d37-439c-bf71-bf137fc4a470"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:59:57.269390 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.267925    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume" pod "2772247b-ee45-43aa-bfbe-ec4697942d72" (UID: "2772247b-ee45-43aa-bfbe-ec4697942d72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:59:57.270680 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.270632    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm" pod "2772247b-ee45-43aa-bfbe-ec4697942d72" (UID: "2772247b-ee45-43aa-bfbe-ec4697942d72"). InnerVolumeSpecName "kube-api-access-p9vcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:57.272350 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.272321    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc" pod "67265b28-0d37-439c-bf71-bf137fc4a470" (UID: "67265b28-0d37-439c-bf71-bf137fc4a470"). InnerVolumeSpecName "kube-api-access-txsmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:57.367203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.367165    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67265b28-0d37-439c-bf71-bf137fc4a470-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.367203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.367199    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2772247b-ee45-43aa-bfbe-ec4697942d72-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.367203 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.367209    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9vcm\" (UniqueName: \"kubernetes.io/projected/2772247b-ee45-43aa-bfbe-ec4697942d72-kube-api-access-p9vcm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.367450 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.367217    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txsmc\" (UniqueName: \"kubernetes.io/projected/67265b28-0d37-439c-bf71-bf137fc4a470-kube-api-access-txsmc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.553903 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.553868    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 21:59:57.554094 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.553931    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 21:59:57.562582 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.562547    4888 generic.go:356] "Generic (PLEG): container finished" podID="67265b28-0d37-439c-bf71-bf137fc4a470" containerID="f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a" exitCode=0
Nov 02 21:59:57.562745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.562592    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"
Nov 02 21:59:57.562745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.562625    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" event={"ID":"67265b28-0d37-439c-bf71-bf137fc4a470","Type":"ContainerDied","Data":"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"}
Nov 02 21:59:57.562745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.562695    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57" event={"ID":"67265b28-0d37-439c-bf71-bf137fc4a470","Type":"ContainerDied","Data":"164bc8f7e6a7db8eecb12b2409c2bccf05eedca6aecd1073ffbff2637fb46c75"}
Nov 02 21:59:57.562745 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.562730    4888 scope.go:122] "RemoveContainer" containerID="f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"
Nov 02 21:59:57.564507 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.564480    4888 generic.go:356] "Generic (PLEG): container finished" podID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerID="e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b" exitCode=0
Nov 02 21:59:57.564622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.564532    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"
Nov 02 21:59:57.564622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.564567    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" event={"ID":"2772247b-ee45-43aa-bfbe-ec4697942d72","Type":"ContainerDied","Data":"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"}
Nov 02 21:59:57.564622 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.564619    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv" event={"ID":"2772247b-ee45-43aa-bfbe-ec4697942d72","Type":"ContainerDied","Data":"d994868d36d114106a20341be7d0a9bf1e1592c82b8c7bdc7de871123e00ae89"}
Nov 02 21:59:57.620277 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.620252    4888 scope.go:122] "RemoveContainer" containerID="f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"
Nov 02 21:59:57.620609 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:57.620586    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a\": container with ID starting with f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a not found: ID does not exist" containerID="f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"
Nov 02 21:59:57.620672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.620620    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a"} err="failed to get container status \"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a\": rpc error: code = NotFound desc = could not find container \"f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a\": container with ID starting with f473ba3e4141eb3d2b62d80edf1647b3e7de98317beecc701366f1ea7020659a not found: ID does not exist"
Nov 02 21:59:57.620672 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.620637    4888 scope.go:122] "RemoveContainer" containerID="e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"
Nov 02 21:59:57.630894 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.630841    4888 scope.go:122] "RemoveContainer" containerID="e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"
Nov 02 21:59:57.631214 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:57.631188    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b\": container with ID starting with e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b not found: ID does not exist" containerID="e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"
Nov 02 21:59:57.631302 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.631225    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b"} err="failed to get container status \"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b\": rpc error: code = NotFound desc = could not find container \"e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b\": container with ID starting with e562e70af04cbdcda899effb8ac17ad2f02fc13b0e4652aa460f965ed34b9e6b not found: ID does not exist"
Nov 02 21:59:57.729890 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.728186    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:59:57.732292 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.732262    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv"]
Nov 02 21:59:57.809368 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.809328    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:59:57.833792 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.833759    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:59:57.837761 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.837734    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57"]
Nov 02 21:59:57.872159 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872115    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872170    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872255    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872364 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872318    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872369    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872399    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872442    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872537 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872496    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.872756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872563    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:57.872756 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872574    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:57.873018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872978    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:57.873018 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.872984    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:59:57.873227 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873069    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\") pod \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\" (UID: \"5ab89e4f-dec4-45c1-baa2-e9943304b23a\") "
Nov 02 21:59:57.873441 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873419    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:57.873641 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873621    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.873722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873650    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.873722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873665    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.873722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873679    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.873722 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.873692    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.874796 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.874767    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:59:57.874985 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.874968    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "kube-api-access-n5tsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:57.875036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.875002    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:57.875036 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.875023    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token" pod "5ab89e4f-dec4-45c1-baa2-e9943304b23a" (UID: "5ab89e4f-dec4-45c1-baa2-e9943304b23a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:57.979207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.977217    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.979207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.977268    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.979207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.977292    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5tsc\" (UniqueName: \"kubernetes.io/projected/5ab89e4f-dec4-45c1-baa2-e9943304b23a-kube-api-access-n5tsc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:57.979207 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:57.977308    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab89e4f-dec4-45c1-baa2-e9943304b23a-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 21:59:58.573785 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.573749    4888 generic.go:356] "Generic (PLEG): container finished" podID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerID="a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36" exitCode=0
Nov 02 21:59:58.574249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.573798    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"
Nov 02 21:59:58.574249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.573827    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" event={"ID":"5ab89e4f-dec4-45c1-baa2-e9943304b23a","Type":"ContainerDied","Data":"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"}
Nov 02 21:59:58.574249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.573913    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2" event={"ID":"5ab89e4f-dec4-45c1-baa2-e9943304b23a","Type":"ContainerDied","Data":"7de459061c993cdd12592b9f21f04af61828897501d1f64572f9dc4c0d957aab"}
Nov 02 21:59:58.574249 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.573935    4888 scope.go:122] "RemoveContainer" containerID="a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"
Nov 02 21:59:58.585566 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.585546    4888 scope.go:122] "RemoveContainer" containerID="a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"
Nov 02 21:59:58.585893 ip-10-0-130-83 kubenswrapper[4888]: E1102 21:59:58.585868    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36\": container with ID starting with a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36 not found: ID does not exist" containerID="a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"
Nov 02 21:59:58.585949 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.585904    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36"} err="failed to get container status \"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36\": rpc error: code = NotFound desc = could not find container \"a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36\": container with ID starting with a1a0eff48846b646da3592ea5200b7c47eaf8238b83780b1efd9e16f77bb5d36 not found: ID does not exist"
Nov 02 21:59:58.601898 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.601859    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" path="/var/lib/kubelet/pods/2772247b-ee45-43aa-bfbe-ec4697942d72/volumes"
Nov 02 21:59:58.602425 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.602410    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" path="/var/lib/kubelet/pods/67265b28-0d37-439c-bf71-bf137fc4a470/volumes"
Nov 02 21:59:58.730021 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.729948    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:59:58.783933 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:58.782760    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2"]
Nov 02 21:59:59.513709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:59.513626    4888 patch_prober.go:29] interesting pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:59:59.513709 ip-10-0-130-83 kubenswrapper[4888]: I1102 21:59:59.513682    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:00:00.161339 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161304    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:00.161798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161785    4888 cpu_manager.go:414] "containerMap: removing container" podUID="67375468-ed62-4a72-9704-ecbb8c507ed0" containerName="main"
Nov 02 22:00:00.161798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161798    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67375468-ed62-4a72-9704-ecbb8c507ed0" containerName="main"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161810    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161815    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161835    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161842    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161874    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161879    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161896    4888 cpu_manager.go:414] "containerMap: removing container" podUID="aff3bdee-9072-4a1e-b88b-86e20015c001" containerName="payload-processing"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161900    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aff3bdee-9072-4a1e-b88b-86e20015c001" containerName="payload-processing"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161908    4888 cpu_manager.go:414] "containerMap: removing container" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing"
Nov 02 22:00:00.161909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161912    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161989    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="aff3bdee-9072-4a1e-b88b-86e20015c001" containerName="payload-processing"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.161997    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="67265b28-0d37-439c-bf71-bf137fc4a470" containerName="payload-processing"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.162007    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="67375468-ed62-4a72-9704-ecbb8c507ed0" containerName="main"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.162013    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" containerName="istio-proxy"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.162019    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.162026    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2772247b-ee45-43aa-bfbe-ec4697942d72" containerName="payload-pre-processing"
Nov 02 22:00:00.162219 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.162032    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.166779 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.166759    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.178251 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.178225    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:00.178729 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.178716    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.178783 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.178731    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d63d53d7-46ef-4203-895b-2c7d4db0945b" containerName="istio-proxy"
Nov 02 22:00:00.182069 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.182049    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.183743 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.183722    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:00.187413 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.187394    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.209136 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.209110    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:00.213573 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.213539    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:00.213699 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.213667    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.218450 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.218429    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.227479 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.227443    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:00.241042 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.241014    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:00.241197 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.241142    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.267505 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.267469    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:00.270226 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.270166    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:00.291676 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.291642    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:00.307358 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307318    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hprwz\" (UniqueName: \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.307536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307368    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.307536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307429    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c8dn\" (UniqueName: \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.307536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307500    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5tm\" (UniqueName: \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.307731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307541    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.307731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307612    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.307731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307665    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqlx8\" (UniqueName: \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.307731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307687    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p5g2\" (UniqueName: \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.308006 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307741    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.308006 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307763    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncwbc\" (UniqueName: \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.308006 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307795    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.308006 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.307818    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.324280 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.319070    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j922b\""
Nov 02 22:00:00.329825 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.329176    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:00.332250 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.332216    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:00.409531 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.409499    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqlx8\" (UniqueName: \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.409702 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.409545    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p5g2\" (UniqueName: \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.409702 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.409621    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.409702 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.409651    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncwbc\" (UniqueName: \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.409946 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.409712    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.410042 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410023    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.410101 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410080    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hprwz\" (UniqueName: \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.410158 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410107    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.410158 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410134    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8dn\" (UniqueName: \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.410269 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410174    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5tm\" (UniqueName: \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.410269 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410220    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.410367 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410323    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410644    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410757    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410974    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.410991    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.411032    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.411691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.411609    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.464147 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.464010    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqlx8\" (UniqueName: \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\") pod \"maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.464799 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.464765    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hprwz\" (UniqueName: \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.464974 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.464927    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncwbc\" (UniqueName: \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\") pod \"maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.465460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.465438    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8dn\" (UniqueName: \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\") pod \"maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.465830 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.465809    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p5g2\" (UniqueName: \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\") pod \"maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.466763 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.466743    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5tm\" (UniqueName: \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\") pod \"maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.480983 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.480643    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:00.497385 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.497291    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:00.507309 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.507102    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:00.543523 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.543216    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:00.568104 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.568060    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:00.574759 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.574700    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:00.624173 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.624070    4888 generic.go:356] "Generic (PLEG): container finished" podID="40bfd517-4735-4472-8a0a-963f57391648" containerID="a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea" exitCode=0
Nov 02 22:00:00.628120 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.627984    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 22:00:00.630191 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.630132    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ab89e4f-dec4-45c1-baa2-e9943304b23a" path="/var/lib/kubelet/pods/5ab89e4f-dec4-45c1-baa2-e9943304b23a/volumes"
Nov 02 22:00:00.630615 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.630588    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerDied","Data":"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"}
Nov 02 22:00:00.630710 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.630643    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4" event={"ID":"40bfd517-4735-4472-8a0a-963f57391648","Type":"ContainerDied","Data":"78f794da366d68f65ab9cf791fc7d2ca95c7fd0c003e1a153a594aa551bb6d1c"}
Nov 02 22:00:00.630710 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.630668    4888 scope.go:122] "RemoveContainer" containerID="a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"
Nov 02 22:00:00.652775 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.647887    4888 scope.go:122] "RemoveContainer" containerID="729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"
Nov 02 22:00:00.666647 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.666403    4888 scope.go:122] "RemoveContainer" containerID="a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"
Nov 02 22:00:00.667153 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:00.667075    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea\": container with ID starting with a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea not found: ID does not exist" containerID="a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"
Nov 02 22:00:00.667378 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.667256    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea"} err="failed to get container status \"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea\": rpc error: code = NotFound desc = could not find container \"a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea\": container with ID starting with a5f8f4c76d9d082bec9a3791f523d9fee8c49c667886463d0a742937b5cbd6ea not found: ID does not exist"
Nov 02 22:00:00.667378 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.667285    4888 scope.go:122] "RemoveContainer" containerID="729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"
Nov 02 22:00:00.667646 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:00.667590    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a\": container with ID starting with 729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a not found: ID does not exist" containerID="729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"
Nov 02 22:00:00.667646 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.667622    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a"} err="failed to get container status \"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a\": rpc error: code = NotFound desc = could not find container \"729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a\": container with ID starting with 729dd3a21b3eb39041ba10332cf2d161c32c6c10729b77a5ac1a69029418320a not found: ID does not exist"
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.717766    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.717819    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.717881    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.717972    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.718073    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.718167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.718133    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.719083 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.718761    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:00.719155 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.719130    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.719199    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.719563    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.719660    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.719746    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.720668    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\") pod \"40bfd517-4735-4472-8a0a-963f57391648\" (UID: \"40bfd517-4735-4472-8a0a-963f57391648\") "
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.720554    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.721339    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.721360    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.721376    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.721391    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bfd517-4735-4472-8a0a-963f57391648-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.721748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.721404    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.724268 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.724231    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:00.725276 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.725241    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:00:00.725562 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.725527    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:00.730096 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.729569    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs" pod "40bfd517-4735-4472-8a0a-963f57391648" (UID: "40bfd517-4735-4472-8a0a-963f57391648"). InnerVolumeSpecName "kube-api-access-jt8vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:00.823029 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.822996    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.823029 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.823025    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bfd517-4735-4472-8a0a-963f57391648-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.823029 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.823037    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt8vs\" (UniqueName: \"kubernetes.io/projected/40bfd517-4735-4472-8a0a-963f57391648-kube-api-access-jt8vs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.823287 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.823047    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bfd517-4735-4472-8a0a-963f57391648-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:00.908810 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.908778    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:00.909903 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:00.909875    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca68326_bb48_4e77_bc54_8d05a2913ba8.slice/crio-d0926e3bbeef2b87616f55ba8499a2dacf88ee72e80d6bd9438d6f297db737e4 WatchSource:0}: Error finding container d0926e3bbeef2b87616f55ba8499a2dacf88ee72e80d6bd9438d6f297db737e4: Status 404 returned error can't find the container with id d0926e3bbeef2b87616f55ba8499a2dacf88ee72e80d6bd9438d6f297db737e4
Nov 02 22:00:00.958192 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:00.958148    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:00.958591 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:00.958567    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584e1c18_3c68_4e04_85a2_0a2b9e72354d.slice/crio-fe5f9b65c23ba325eb693f078406cfdc688922ae6363ded93f2d3b720c3dc656 WatchSource:0}: Error finding container fe5f9b65c23ba325eb693f078406cfdc688922ae6363ded93f2d3b720c3dc656: Status 404 returned error can't find the container with id fe5f9b65c23ba325eb693f078406cfdc688922ae6363ded93f2d3b720c3dc656
Nov 02 22:00:01.002238 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.002213    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:01.004216 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:01.004148    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf788d02c_acc8_49e5_ab6e_3087c6f0f6d3.slice/crio-430cf8fbeae0c133d438c9faf26aedcf7573813aa5279521443d98fbba4da5cc WatchSource:0}: Error finding container 430cf8fbeae0c133d438c9faf26aedcf7573813aa5279521443d98fbba4da5cc: Status 404 returned error can't find the container with id 430cf8fbeae0c133d438c9faf26aedcf7573813aa5279521443d98fbba4da5cc
Nov 02 22:00:01.008071 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.008048    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:01.010438 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:01.010409    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2f7635_8670_46f4_a369_6bd8f552ddd1.slice/crio-05a7f90ebd1c5476cd7d13dba1549dfa08c9ddee3e8a2e638ca690909ef31cc0 WatchSource:0}: Error finding container 05a7f90ebd1c5476cd7d13dba1549dfa08c9ddee3e8a2e638ca690909ef31cc0: Status 404 returned error can't find the container with id 05a7f90ebd1c5476cd7d13dba1549dfa08c9ddee3e8a2e638ca690909ef31cc0
Nov 02 22:00:01.037898 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.037871    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:01.047744 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.047716    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:01.113817 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:01.113779    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb557a0b0_277a_4e57_9646_ccf99619b700.slice/crio-80f9bd580df0c54888a6e01c83551a96f6589212d4b104b421fc2355c1f5bc4c WatchSource:0}: Error finding container 80f9bd580df0c54888a6e01c83551a96f6589212d4b104b421fc2355c1f5bc4c: Status 404 returned error can't find the container with id 80f9bd580df0c54888a6e01c83551a96f6589212d4b104b421fc2355c1f5bc4c
Nov 02 22:00:01.114790 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:01.114639    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad514094_ac08_456c_9b01_0c4f4d6242cd.slice/crio-9c22648f433cabf00cfe21ece91239d379bf85acccf5546e9b293da841877798 WatchSource:0}: Error finding container 9c22648f433cabf00cfe21ece91239d379bf85acccf5546e9b293da841877798: Status 404 returned error can't find the container with id 9c22648f433cabf00cfe21ece91239d379bf85acccf5546e9b293da841877798
Nov 02 22:00:01.632699 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.632668    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerStarted","Data":"e16772437ba76a5a9ddf2e1c6c274139d6395175194fe11748c17ca95ff539da"}
Nov 02 22:00:01.633144 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.632733    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerStarted","Data":"fe5f9b65c23ba325eb693f078406cfdc688922ae6363ded93f2d3b720c3dc656"}
Nov 02 22:00:01.633699 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.633681    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"
Nov 02 22:00:01.635251 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.635221    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerStarted","Data":"e0a8f6f3995377119eccec62306a99b3f7ab62b747f2d7931d4966732893e55d"}
Nov 02 22:00:01.635360 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.635267    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerStarted","Data":"d0926e3bbeef2b87616f55ba8499a2dacf88ee72e80d6bd9438d6f297db737e4"}
Nov 02 22:00:01.636813 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.636792    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerStarted","Data":"60e6d49652d7b41755c8ef813c00702146dd3d241e0b072aae2b7f722f6c022d"}
Nov 02 22:00:01.636919 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.636830    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerStarted","Data":"430cf8fbeae0c133d438c9faf26aedcf7573813aa5279521443d98fbba4da5cc"}
Nov 02 22:00:01.638042 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.638014    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerStarted","Data":"80f9bd580df0c54888a6e01c83551a96f6589212d4b104b421fc2355c1f5bc4c"}
Nov 02 22:00:01.639665 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.639641    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerStarted","Data":"05a7f90ebd1c5476cd7d13dba1549dfa08c9ddee3e8a2e638ca690909ef31cc0"}
Nov 02 22:00:01.641153 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.641131    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerStarted","Data":"dc9ec93838e9dde18b7d850c9b616b7b7d0e297be3bf43cc601771937321f6f1"}
Nov 02 22:00:01.641236 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.641172    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerStarted","Data":"9c22648f433cabf00cfe21ece91239d379bf85acccf5546e9b293da841877798"}
Nov 02 22:00:01.759286 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.759230    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" podStartSLOduration=1.759216698 podStartE2EDuration="1.759216698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:01.757269189 +0000 UTC m=+2831.301061110" watchObservedRunningTime="2026-11-02 22:00:01.759216698 +0000 UTC m=+2831.303008613"
Nov 02 22:00:01.864417 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.864376    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 22:00:01.869896 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.869841    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4"]
Nov 02 22:00:01.975642 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:01.975581    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" podStartSLOduration=1.975566857 podStartE2EDuration="1.975566857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:01.974402074 +0000 UTC m=+2831.518194014" watchObservedRunningTime="2026-11-02 22:00:01.975566857 +0000 UTC m=+2831.519358770"
Nov 02 22:00:02.084147 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.084102    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" podStartSLOduration=2.084088818 podStartE2EDuration="2.084088818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:02.081886856 +0000 UTC m=+2831.625678770" watchObservedRunningTime="2026-11-02 22:00:02.084088818 +0000 UTC m=+2831.627880732"
Nov 02 22:00:02.190710 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.190654    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" podStartSLOduration=2.190639715 podStartE2EDuration="2.190639715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:02.188284187 +0000 UTC m=+2831.732076114" watchObservedRunningTime="2026-11-02 22:00:02.190639715 +0000 UTC m=+2831.734431791"
Nov 02 22:00:02.299979 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.299914    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" podStartSLOduration=2.299892472 podStartE2EDuration="2.299892472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:02.296205801 +0000 UTC m=+2831.839997716" watchObservedRunningTime="2026-11-02 22:00:02.299892472 +0000 UTC m=+2831.843684388"
Nov 02 22:00:02.633898 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.633393    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40bfd517-4735-4472-8a0a-963f57391648" path="/var/lib/kubelet/pods/40bfd517-4735-4472-8a0a-963f57391648/volumes"
Nov 02 22:00:02.653406 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.653373    4888 generic.go:356] "Generic (PLEG): container finished" podID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerID="e16772437ba76a5a9ddf2e1c6c274139d6395175194fe11748c17ca95ff539da" exitCode=60
Nov 02 22:00:02.653621 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.653458    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerDied","Data":"e16772437ba76a5a9ddf2e1c6c274139d6395175194fe11748c17ca95ff539da"}
Nov 02 22:00:02.653768 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.653743    4888 scope.go:122] "RemoveContainer" containerID="e16772437ba76a5a9ddf2e1c6c274139d6395175194fe11748c17ca95ff539da"
Nov 02 22:00:02.655345 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.655250    4888 generic.go:356] "Generic (PLEG): container finished" podID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerID="e0a8f6f3995377119eccec62306a99b3f7ab62b747f2d7931d4966732893e55d" exitCode=60
Nov 02 22:00:02.655418 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.655383    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerDied","Data":"e0a8f6f3995377119eccec62306a99b3f7ab62b747f2d7931d4966732893e55d"}
Nov 02 22:00:02.655561 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.655536    4888 scope.go:122] "RemoveContainer" containerID="e0a8f6f3995377119eccec62306a99b3f7ab62b747f2d7931d4966732893e55d"
Nov 02 22:00:02.657428 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.657406    4888 generic.go:356] "Generic (PLEG): container finished" podID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerID="60e6d49652d7b41755c8ef813c00702146dd3d241e0b072aae2b7f722f6c022d" exitCode=60
Nov 02 22:00:02.657538 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.657502    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerDied","Data":"60e6d49652d7b41755c8ef813c00702146dd3d241e0b072aae2b7f722f6c022d"}
Nov 02 22:00:02.657662 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.657649    4888 scope.go:122] "RemoveContainer" containerID="60e6d49652d7b41755c8ef813c00702146dd3d241e0b072aae2b7f722f6c022d"
Nov 02 22:00:02.659059 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.659036    4888 generic.go:356] "Generic (PLEG): container finished" podID="b557a0b0-277a-4e57-9646-ccf99619b700" containerID="0849369bc65039c597b51bf89a3e0a0233521e57613c51a995a6e7f1d7d45cae" exitCode=60
Nov 02 22:00:02.659157 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.659105    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerDied","Data":"0849369bc65039c597b51bf89a3e0a0233521e57613c51a995a6e7f1d7d45cae"}
Nov 02 22:00:02.659261 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.659244    4888 scope.go:122] "RemoveContainer" containerID="0849369bc65039c597b51bf89a3e0a0233521e57613c51a995a6e7f1d7d45cae"
Nov 02 22:00:02.660620 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.660601    4888 generic.go:356] "Generic (PLEG): container finished" podID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerID="ae6fe785b04c32b1795897f809b8224fb3b49afe641f995da43a2185bd9e8675" exitCode=60
Nov 02 22:00:02.660713 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.660648    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerDied","Data":"ae6fe785b04c32b1795897f809b8224fb3b49afe641f995da43a2185bd9e8675"}
Nov 02 22:00:02.660937 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.660917    4888 scope.go:122] "RemoveContainer" containerID="ae6fe785b04c32b1795897f809b8224fb3b49afe641f995da43a2185bd9e8675"
Nov 02 22:00:02.662244 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.662210    4888 generic.go:356] "Generic (PLEG): container finished" podID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerID="dc9ec93838e9dde18b7d850c9b616b7b7d0e297be3bf43cc601771937321f6f1" exitCode=60
Nov 02 22:00:02.662343 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.662259    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerDied","Data":"dc9ec93838e9dde18b7d850c9b616b7b7d0e297be3bf43cc601771937321f6f1"}
Nov 02 22:00:02.662451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:02.662437    4888 scope.go:122] "RemoveContainer" containerID="dc9ec93838e9dde18b7d850c9b616b7b7d0e297be3bf43cc601771937321f6f1"
Nov 02 22:00:03.670791 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.670753    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerStarted","Data":"de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"}
Nov 02 22:00:03.672781 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.672755    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerStarted","Data":"31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"}
Nov 02 22:00:03.674534 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.674512    4888 generic.go:356] "Generic (PLEG): container finished" podID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerID="137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc" exitCode=60
Nov 02 22:00:03.674660 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.674595    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerDied","Data":"137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc"}
Nov 02 22:00:03.674660 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.674652    4888 scope.go:122] "RemoveContainer" containerID="dc9ec93838e9dde18b7d850c9b616b7b7d0e297be3bf43cc601771937321f6f1"
Nov 02 22:00:03.674842 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.674824    4888 scope.go:122] "RemoveContainer" containerID="137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc"
Nov 02 22:00:03.675161 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:03.675136    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28_odh-ai-gateway-infra(ad514094-ac08-456c-9b01-0c4f4d6242cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd"
Nov 02 22:00:03.676754 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.676734    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerStarted","Data":"94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"}
Nov 02 22:00:03.678499 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.678477    4888 generic.go:356] "Generic (PLEG): container finished" podID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerID="2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b" exitCode=60
Nov 02 22:00:03.678598 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.678556    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerDied","Data":"2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b"}
Nov 02 22:00:03.678757 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.678740    4888 scope.go:122] "RemoveContainer" containerID="2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b"
Nov 02 22:00:03.679067 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:03.679043    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d_odh-ai-gateway-infra(6ca68326-bb48-4e77-bc54-8d05a2913ba8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8"
Nov 02 22:00:03.680627 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.680605    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerStarted","Data":"74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"}
Nov 02 22:00:03.693773 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.693743    4888 scope.go:122] "RemoveContainer" containerID="e0a8f6f3995377119eccec62306a99b3f7ab62b747f2d7931d4966732893e55d"
Nov 02 22:00:03.779337 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:03.779287    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" podStartSLOduration=3.7792729229999997 podStartE2EDuration="3.779272923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:03.777857252 +0000 UTC m=+2833.321649159" watchObservedRunningTime="2026-11-02 22:00:03.779272923 +0000 UTC m=+2833.323064839"
Nov 02 22:00:04.786367 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.786340    4888 scope.go:122] "RemoveContainer" containerID="137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc"
Nov 02 22:00:04.786840 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.786607    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28_odh-ai-gateway-infra(ad514094-ac08-456c-9b01-0c4f4d6242cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd"
Nov 02 22:00:04.788526 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.788500    4888 generic.go:356] "Generic (PLEG): container finished" podID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerID="94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037" exitCode=60
Nov 02 22:00:04.788657 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.788580    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerDied","Data":"94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"}
Nov 02 22:00:04.788657 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.788647    4888 scope.go:122] "RemoveContainer" containerID="e16772437ba76a5a9ddf2e1c6c274139d6395175194fe11748c17ca95ff539da"
Nov 02 22:00:04.788798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.788783    4888 scope.go:122] "RemoveContainer" containerID="94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"
Nov 02 22:00:04.789065 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.789040    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj_odh-ai-gateway-infra(584e1c18-3c68-4e04-85a2-0a2b9e72354d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d"
Nov 02 22:00:04.790447 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.790429    4888 scope.go:122] "RemoveContainer" containerID="2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b"
Nov 02 22:00:04.790690 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.790654    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d_odh-ai-gateway-infra(6ca68326-bb48-4e77-bc54-8d05a2913ba8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8"
Nov 02 22:00:04.792243 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.792220    4888 generic.go:356] "Generic (PLEG): container finished" podID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerID="74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473" exitCode=60
Nov 02 22:00:04.792344 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.792294    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerDied","Data":"74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"}
Nov 02 22:00:04.792489 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.792474    4888 scope.go:122] "RemoveContainer" containerID="74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"
Nov 02 22:00:04.792732 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.792711    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq_odh-ai-gateway-infra(f788d02c-acc8-49e5-ab6e-3087c6f0f6d3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3"
Nov 02 22:00:04.794461 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.794436    4888 generic.go:356] "Generic (PLEG): container finished" podID="b557a0b0-277a-4e57-9646-ccf99619b700" containerID="de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22" exitCode=60
Nov 02 22:00:04.794554 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.794534    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerDied","Data":"de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"}
Nov 02 22:00:04.794733 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.794715    4888 scope.go:122] "RemoveContainer" containerID="de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"
Nov 02 22:00:04.795034 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.795011    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn_odh-ai-gateway-infra(b557a0b0-277a-4e57-9646-ccf99619b700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" podUID="b557a0b0-277a-4e57-9646-ccf99619b700"
Nov 02 22:00:04.797019 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.796995    4888 generic.go:356] "Generic (PLEG): container finished" podID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerID="31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963" exitCode=60
Nov 02 22:00:04.797118 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.797044    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerDied","Data":"31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"}
Nov 02 22:00:04.797314 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.797292    4888 scope.go:122] "RemoveContainer" containerID="31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"
Nov 02 22:00:04.797496 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:04.797481    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m_odh-ai-gateway-infra(cd2f7635-8670-46f4-a369-6bd8f552ddd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1"
Nov 02 22:00:04.802570 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.802551    4888 scope.go:122] "RemoveContainer" containerID="60e6d49652d7b41755c8ef813c00702146dd3d241e0b072aae2b7f722f6c022d"
Nov 02 22:00:04.816017 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.815991    4888 scope.go:122] "RemoveContainer" containerID="0849369bc65039c597b51bf89a3e0a0233521e57613c51a995a6e7f1d7d45cae"
Nov 02 22:00:04.829450 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:04.829418    4888 scope.go:122] "RemoveContainer" containerID="ae6fe785b04c32b1795897f809b8224fb3b49afe641f995da43a2185bd9e8675"
Nov 02 22:00:05.810306 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:05.810268    4888 scope.go:122] "RemoveContainer" containerID="94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"
Nov 02 22:00:05.810788 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:05.810644    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj_odh-ai-gateway-infra(584e1c18-3c68-4e04-85a2-0a2b9e72354d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d"
Nov 02 22:00:05.811908 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:05.811886    4888 scope.go:122] "RemoveContainer" containerID="74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"
Nov 02 22:00:05.812184 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:05.812163    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq_odh-ai-gateway-infra(f788d02c-acc8-49e5-ab6e-3087c6f0f6d3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3"
Nov 02 22:00:05.813368 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:05.813346    4888 scope.go:122] "RemoveContainer" containerID="de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"
Nov 02 22:00:05.813647 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:05.813622    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn_odh-ai-gateway-infra(b557a0b0-277a-4e57-9646-ccf99619b700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" podUID="b557a0b0-277a-4e57-9646-ccf99619b700"
Nov 02 22:00:05.814857 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:05.814828    4888 scope.go:122] "RemoveContainer" containerID="31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"
Nov 02 22:00:05.815062 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:05.815045    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m_odh-ai-gateway-infra(cd2f7635-8670-46f4-a369-6bd8f552ddd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1"
Nov 02 22:00:13.587726 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.587685    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"]
Nov 02 22:00:13.588303 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588288    4888 cpu_manager.go:414] "containerMap: removing container" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.588349 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588304    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.588349 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588320    4888 cpu_manager.go:414] "containerMap: removing container" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.588349 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588325    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.588459 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588415    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.588459 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.588426    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="40bfd517-4735-4472-8a0a-963f57391648" containerName="istio-proxy"
Nov 02 22:00:13.593250 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.593225    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.605739 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.605703    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.605933 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.605905    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zkd5\" (UniqueName: \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.606085 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.606020    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.642939 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.642902    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 22:00:13.695519 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.695482    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"]
Nov 02 22:00:13.706790 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.706746    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.706977 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.706878    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.707064 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.707009    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zkd5\" (UniqueName: \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.707708 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.707680    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.709260 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.709231    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.760972 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.760940    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zkd5\" (UniqueName: \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\") pod \"maas-api-revoke-sub-9adafe35-cc4zf\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:13.909315 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:13.909275    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:14.288661 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:14.288629    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"]
Nov 02 22:00:14.289299 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:00:14.289273    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe8f6b2_4ecd_4282_9cfb_da6b0e6b04a5.slice/crio-3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92 WatchSource:0}: Error finding container 3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92: Status 404 returned error can't find the container with id 3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92
Nov 02 22:00:14.874647 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:14.874555    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf" event={"ID":"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5","Type":"ContainerStarted","Data":"ee9ff71aac7c44d33e6d3de29ee9ce75d7ca3357653f2f1ef567d395014653ce"}
Nov 02 22:00:14.874647 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:14.874625    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf" event={"ID":"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5","Type":"ContainerStarted","Data":"3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92"}
Nov 02 22:00:15.031244 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:15.031180    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf" podStartSLOduration=2.03116368 podStartE2EDuration="2.03116368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:14.983277768 +0000 UTC m=+2844.527069684" watchObservedRunningTime="2026-11-02 22:00:15.03116368 +0000 UTC m=+2844.574955596"
Nov 02 22:00:15.593799 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:15.593704    4888 scope.go:122] "RemoveContainer" containerID="2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b"
Nov 02 22:00:15.593799 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:15.593750    4888 scope.go:122] "RemoveContainer" containerID="de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"
Nov 02 22:00:15.883284 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:15.883197    4888 generic.go:356] "Generic (PLEG): container finished" podID="ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" containerID="ee9ff71aac7c44d33e6d3de29ee9ce75d7ca3357653f2f1ef567d395014653ce" exitCode=0
Nov 02 22:00:15.883284 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:15.883266    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf" event={"ID":"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5","Type":"ContainerDied","Data":"ee9ff71aac7c44d33e6d3de29ee9ce75d7ca3357653f2f1ef567d395014653ce"}
Nov 02 22:00:16.593210 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.593173    4888 scope.go:122] "RemoveContainer" containerID="74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"
Nov 02 22:00:16.593381 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.593301    4888 scope.go:122] "RemoveContainer" containerID="31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"
Nov 02 22:00:16.891556 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.891513    4888 generic.go:356] "Generic (PLEG): container finished" podID="b557a0b0-277a-4e57-9646-ccf99619b700" containerID="ffbee99096281ae3f99ae3ad1deb4456376a53e84113be8cdeaeb4d71dfb8b95" exitCode=60
Nov 02 22:00:16.892070 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.891591    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerDied","Data":"ffbee99096281ae3f99ae3ad1deb4456376a53e84113be8cdeaeb4d71dfb8b95"}
Nov 02 22:00:16.892070 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.891668    4888 scope.go:122] "RemoveContainer" containerID="de542feac672555da6daeffbeaaa695de7f11a3e97efd2d92ea7a2903f4baa22"
Nov 02 22:00:16.892216 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.892198    4888 scope.go:122] "RemoveContainer" containerID="ffbee99096281ae3f99ae3ad1deb4456376a53e84113be8cdeaeb4d71dfb8b95"
Nov 02 22:00:16.892556 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:16.892532    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn_odh-ai-gateway-infra(b557a0b0-277a-4e57-9646-ccf99619b700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" podUID="b557a0b0-277a-4e57-9646-ccf99619b700"
Nov 02 22:00:16.893964 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.893941    4888 generic.go:356] "Generic (PLEG): container finished" podID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerID="1ad6ad878c4925ba19a9e3a27c5d746cd23909ad07ab474af0e4ab76ba6c1d0d" exitCode=60
Nov 02 22:00:16.894058 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.894014    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerDied","Data":"1ad6ad878c4925ba19a9e3a27c5d746cd23909ad07ab474af0e4ab76ba6c1d0d"}
Nov 02 22:00:16.894381 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.894366    4888 scope.go:122] "RemoveContainer" containerID="1ad6ad878c4925ba19a9e3a27c5d746cd23909ad07ab474af0e4ab76ba6c1d0d"
Nov 02 22:00:16.894638 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:16.894621    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d_odh-ai-gateway-infra(6ca68326-bb48-4e77-bc54-8d05a2913ba8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8"
Nov 02 22:00:16.906387 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:16.906364    4888 scope.go:122] "RemoveContainer" containerID="2830f2bdf7cf06d39612907ddb9eb93a91673794362a1b2e4938f5d5ed10306b"
Nov 02 22:00:17.050483 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.050453    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:17.147127 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.147089    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\") pod \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") "
Nov 02 22:00:17.147300 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.147268    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\") pod \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") "
Nov 02 22:00:17.147420 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.147404    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\" (UniqueName: \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\") pod \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\" (UID: \"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5\") "
Nov 02 22:00:17.147700 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.147670    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca" pod "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" (UID: "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:17.147915 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.147897    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-openshift-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:17.149429 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.149404    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca" pod "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" (UID: "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:00:17.149837 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.149818    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5" pod "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" (UID: "ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5"). InnerVolumeSpecName "kube-api-access-4zkd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:17.249891 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.249834    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zkd5\" (UniqueName: \"kubernetes.io/projected/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-kube-api-access-4zkd5\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:17.249891 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.249893    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5-maas-api-xks-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:17.593623 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.593585    4888 scope.go:122] "RemoveContainer" containerID="137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc"
Nov 02 22:00:17.905778 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.905736    4888 generic.go:356] "Generic (PLEG): container finished" podID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerID="0f7d1f7e194e7c6aa1181c3321d6a786845b6e8653bf84ae3114a611107b5671" exitCode=60
Nov 02 22:00:17.906247 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.905812    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerDied","Data":"0f7d1f7e194e7c6aa1181c3321d6a786845b6e8653bf84ae3114a611107b5671"}
Nov 02 22:00:17.906247 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.905903    4888 scope.go:122] "RemoveContainer" containerID="74fce6467eebf19bb5768eb3623dea9457fd7f1ebc7d853f92bb5c3765619473"
Nov 02 22:00:17.906377 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.906360    4888 scope.go:122] "RemoveContainer" containerID="0f7d1f7e194e7c6aa1181c3321d6a786845b6e8653bf84ae3114a611107b5671"
Nov 02 22:00:17.906926 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:17.906791    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq_odh-ai-gateway-infra(f788d02c-acc8-49e5-ab6e-3087c6f0f6d3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3"
Nov 02 22:00:17.909432 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.909407    4888 generic.go:356] "Generic (PLEG): container finished" podID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerID="d063fcd44392383dea33f56b6dbbe1ea8de04f462cc7cf1bb635db340c2ff1e6" exitCode=60
Nov 02 22:00:17.909542 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.909477    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerDied","Data":"d063fcd44392383dea33f56b6dbbe1ea8de04f462cc7cf1bb635db340c2ff1e6"}
Nov 02 22:00:17.909900 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.909883    4888 scope.go:122] "RemoveContainer" containerID="d063fcd44392383dea33f56b6dbbe1ea8de04f462cc7cf1bb635db340c2ff1e6"
Nov 02 22:00:17.910206 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:17.910187    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m_odh-ai-gateway-infra(cd2f7635-8670-46f4-a369-6bd8f552ddd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1"
Nov 02 22:00:17.910932 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.910915    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"
Nov 02 22:00:17.911045 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.910943    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf" event={"ID":"ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5","Type":"ContainerDied","Data":"3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92"}
Nov 02 22:00:17.911045 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.910993    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ffc559dabf92dc105614719d4452601591c73fcd45e6ef5d06b6e73db0c8d92"
Nov 02 22:00:17.919972 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:17.919950    4888 scope.go:122] "RemoveContainer" containerID="31b85d04edf12aa0fa21e49b9aea62a8db28edde7e6e14c906380a3c810c8963"
Nov 02 22:00:18.016133 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.016085    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:18.124649 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.124618    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:18.186189 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.186165    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:18.261767 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.261735    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\" (UniqueName: \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\") pod \"b557a0b0-277a-4e57-9646-ccf99619b700\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") "
Nov 02 22:00:18.261955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.261797    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\") pod \"b557a0b0-277a-4e57-9646-ccf99619b700\" (UID: \"b557a0b0-277a-4e57-9646-ccf99619b700\") "
Nov 02 22:00:18.262244 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.262213    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca" pod "b557a0b0-277a-4e57-9646-ccf99619b700" (UID: "b557a0b0-277a-4e57-9646-ccf99619b700"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:18.264255 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.264229    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz" pod "b557a0b0-277a-4e57-9646-ccf99619b700" (UID: "b557a0b0-277a-4e57-9646-ccf99619b700"). InnerVolumeSpecName "kube-api-access-hprwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:18.290512 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.290485    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:18.363300 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363263    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\") pod \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") "
Nov 02 22:00:18.363455 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363377    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\" (UniqueName: \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\") pod \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\" (UID: \"6ca68326-bb48-4e77-bc54-8d05a2913ba8\") "
Nov 02 22:00:18.363640 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363619    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca" pod "6ca68326-bb48-4e77-bc54-8d05a2913ba8" (UID: "6ca68326-bb48-4e77-bc54-8d05a2913ba8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:18.363793 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363773    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca68326-bb48-4e77-bc54-8d05a2913ba8-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:18.363841 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363799    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hprwz\" (UniqueName: \"kubernetes.io/projected/b557a0b0-277a-4e57-9646-ccf99619b700-kube-api-access-hprwz\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:18.363841 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.363810    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b557a0b0-277a-4e57-9646-ccf99619b700-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:18.365560 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.365541    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8" pod "6ca68326-bb48-4e77-bc54-8d05a2913ba8" (UID: "6ca68326-bb48-4e77-bc54-8d05a2913ba8"). InnerVolumeSpecName "kube-api-access-hqlx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:18.465092 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.465005    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqlx8\" (UniqueName: \"kubernetes.io/projected/6ca68326-bb48-4e77-bc54-8d05a2913ba8-kube-api-access-hqlx8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:18.594091 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.594056    4888 scope.go:122] "RemoveContainer" containerID="94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"
Nov 02 22:00:18.919882 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.919822    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"
Nov 02 22:00:18.919882 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.919835    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d" event={"ID":"6ca68326-bb48-4e77-bc54-8d05a2913ba8","Type":"ContainerDied","Data":"d0926e3bbeef2b87616f55ba8499a2dacf88ee72e80d6bd9438d6f297db737e4"}
Nov 02 22:00:18.920333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.919915    4888 scope.go:122] "RemoveContainer" containerID="1ad6ad878c4925ba19a9e3a27c5d746cd23909ad07ab474af0e4ab76ba6c1d0d"
Nov 02 22:00:18.923712 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.923691    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"
Nov 02 22:00:18.923889 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.923725    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn" event={"ID":"b557a0b0-277a-4e57-9646-ccf99619b700","Type":"ContainerDied","Data":"80f9bd580df0c54888a6e01c83551a96f6589212d4b104b421fc2355c1f5bc4c"}
Nov 02 22:00:18.927933 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.927906    4888 generic.go:356] "Generic (PLEG): container finished" podID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerID="040c967085aba3d358874c5e33122e9b78a695515a3e5851f2df73b33aba1112" exitCode=60
Nov 02 22:00:18.928046 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.927979    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerDied","Data":"040c967085aba3d358874c5e33122e9b78a695515a3e5851f2df73b33aba1112"}
Nov 02 22:00:18.928461 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.928439    4888 scope.go:122] "RemoveContainer" containerID="040c967085aba3d358874c5e33122e9b78a695515a3e5851f2df73b33aba1112"
Nov 02 22:00:18.928817 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:18.928795    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28_odh-ai-gateway-infra(ad514094-ac08-456c-9b01-0c4f4d6242cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd"
Nov 02 22:00:18.932911 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.932892    4888 scope.go:122] "RemoveContainer" containerID="ffbee99096281ae3f99ae3ad1deb4456376a53e84113be8cdeaeb4d71dfb8b95"
Nov 02 22:00:18.941697 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:18.941679    4888 scope.go:122] "RemoveContainer" containerID="137bd0f9908a442e0e6e503ddb397d25e955c51e60777777096ca3a0935f94fc"
Nov 02 22:00:19.024129 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.024038    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:19.081171 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.081130    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:19.084103 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.084079    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4e2b27-29894280-rt88d"]
Nov 02 22:00:19.184416 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.184386    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:19.190497 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.190467    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:19.195759 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.195729    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f22129-29894280-nqfcn"]
Nov 02 22:00:19.203609 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.203578    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:19.275361 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.275284    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\" (UniqueName: \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\") pod \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") "
Nov 02 22:00:19.275361 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.275342    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\") pod \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\" (UID: \"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3\") "
Nov 02 22:00:19.276097 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.276065    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca" pod "f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" (UID: "f788d02c-acc8-49e5-ab6e-3087c6f0f6d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:19.278302 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.278279    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc" pod "f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" (UID: "f788d02c-acc8-49e5-ab6e-3087c6f0f6d3"). InnerVolumeSpecName "kube-api-access-ncwbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:19.309557 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.309533    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:19.376924 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.376890    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\") pod \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") "
Nov 02 22:00:19.377089 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.377036    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\" (UniqueName: \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\") pod \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\" (UID: \"cd2f7635-8670-46f4-a369-6bd8f552ddd1\") "
Nov 02 22:00:19.377265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.377243    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca" pod "cd2f7635-8670-46f4-a369-6bd8f552ddd1" (UID: "cd2f7635-8670-46f4-a369-6bd8f552ddd1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:19.377445 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.377430    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncwbc\" (UniqueName: \"kubernetes.io/projected/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-kube-api-access-ncwbc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:19.377479 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.377448    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:19.377479 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.377459    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f7635-8670-46f4-a369-6bd8f552ddd1-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:19.379227 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.379208    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm" pod "cd2f7635-8670-46f4-a369-6bd8f552ddd1" (UID: "cd2f7635-8670-46f4-a369-6bd8f552ddd1"). InnerVolumeSpecName "kube-api-access-vl5tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:19.478698 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.478662    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl5tm\" (UniqueName: \"kubernetes.io/projected/cd2f7635-8670-46f4-a369-6bd8f552ddd1-kube-api-access-vl5tm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:19.937146 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.937111    4888 generic.go:356] "Generic (PLEG): container finished" podID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerID="03fe8a09ec12ed50f53fae22de6f7a3db10175b6e9f7a95690bc3cce3c85e9eb" exitCode=60
Nov 02 22:00:19.937655 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.937186    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerDied","Data":"03fe8a09ec12ed50f53fae22de6f7a3db10175b6e9f7a95690bc3cce3c85e9eb"}
Nov 02 22:00:19.937655 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.937250    4888 scope.go:122] "RemoveContainer" containerID="94dc301389e5feb6a0330b7e01c7485a22643299e0a25ba84e330c8c0913b037"
Nov 02 22:00:19.937795 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.937736    4888 scope.go:122] "RemoveContainer" containerID="03fe8a09ec12ed50f53fae22de6f7a3db10175b6e9f7a95690bc3cce3c85e9eb"
Nov 02 22:00:19.938222 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:19.938195    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj_odh-ai-gateway-infra(584e1c18-3c68-4e04-85a2-0a2b9e72354d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d"
Nov 02 22:00:19.940490 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.940466    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"
Nov 02 22:00:19.940599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.940486    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq" event={"ID":"f788d02c-acc8-49e5-ab6e-3087c6f0f6d3","Type":"ContainerDied","Data":"430cf8fbeae0c133d438c9faf26aedcf7573813aa5279521443d98fbba4da5cc"}
Nov 02 22:00:19.943380 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.943346    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m" event={"ID":"cd2f7635-8670-46f4-a369-6bd8f552ddd1","Type":"ContainerDied","Data":"05a7f90ebd1c5476cd7d13dba1549dfa08c9ddee3e8a2e638ca690909ef31cc0"}
Nov 02 22:00:19.943511 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.943383    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"
Nov 02 22:00:19.961069 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.961043    4888 scope.go:122] "RemoveContainer" containerID="0f7d1f7e194e7c6aa1181c3321d6a786845b6e8653bf84ae3114a611107b5671"
Nov 02 22:00:19.970541 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:19.970516    4888 scope.go:122] "RemoveContainer" containerID="d063fcd44392383dea33f56b6dbbe1ea8de04f462cc7cf1bb635db340c2ff1e6"
Nov 02 22:00:20.012281 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:20.012235    4888 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf788d02c_acc8_49e5_ab6e_3087c6f0f6d3.slice\": RecentStats: unable to find data in memory cache]"
Nov 02 22:00:20.155985 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.155951    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:20.163819 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.163787    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c1243c-29894280-rlcdq"]
Nov 02 22:00:20.272863 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.271115    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:20.274611 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.274582    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcb857-29894280-xfq2m"]
Nov 02 22:00:20.313037 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.313005    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:20.485387 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.485357    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:20.590376 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.590285    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\") pod \"ad514094-ac08-456c-9b01-0c4f4d6242cd\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") "
Nov 02 22:00:20.590376 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.590339    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\" (UniqueName: \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\") pod \"ad514094-ac08-456c-9b01-0c4f4d6242cd\" (UID: \"ad514094-ac08-456c-9b01-0c4f4d6242cd\") "
Nov 02 22:00:20.591123 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.591093    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca" pod "ad514094-ac08-456c-9b01-0c4f4d6242cd" (UID: "ad514094-ac08-456c-9b01-0c4f4d6242cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:20.593050 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.593012    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2" pod "ad514094-ac08-456c-9b01-0c4f4d6242cd" (UID: "ad514094-ac08-456c-9b01-0c4f4d6242cd"). InnerVolumeSpecName "kube-api-access-9p5g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:20.608885 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.608726    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" path="/var/lib/kubelet/pods/6ca68326-bb48-4e77-bc54-8d05a2913ba8/volumes"
Nov 02 22:00:20.609381 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.609362    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" path="/var/lib/kubelet/pods/b557a0b0-277a-4e57-9646-ccf99619b700/volumes"
Nov 02 22:00:20.610046 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.609898    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" path="/var/lib/kubelet/pods/cd2f7635-8670-46f4-a369-6bd8f552ddd1/volumes"
Nov 02 22:00:20.610838 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.610818    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" path="/var/lib/kubelet/pods/f788d02c-acc8-49e5-ab6e-3087c6f0f6d3/volumes"
Nov 02 22:00:20.692463 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.692418    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad514094-ac08-456c-9b01-0c4f4d6242cd-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:20.692641 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.692473    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p5g2\" (UniqueName: \"kubernetes.io/projected/ad514094-ac08-456c-9b01-0c4f4d6242cd-kube-api-access-9p5g2\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:20.954835 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.954808    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"
Nov 02 22:00:20.955305 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.954836    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28" event={"ID":"ad514094-ac08-456c-9b01-0c4f4d6242cd","Type":"ContainerDied","Data":"9c22648f433cabf00cfe21ece91239d379bf85acccf5546e9b293da841877798"}
Nov 02 22:00:20.955305 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:20.954904    4888 scope.go:122] "RemoveContainer" containerID="040c967085aba3d358874c5e33122e9b78a695515a3e5851f2df73b33aba1112"
Nov 02 22:00:21.055764 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.055729    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:21.118129 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.118096    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:21.122491 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.122460    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-11537e-29894280-z2r28"]
Nov 02 22:00:21.207090 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.207022    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:21.298899 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.298829    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\" (UniqueName: \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\") pod \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") "
Nov 02 22:00:21.299125 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.299032    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\") pod \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\" (UID: \"584e1c18-3c68-4e04-85a2-0a2b9e72354d\") "
Nov 02 22:00:21.299416 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.299391    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca" pod "584e1c18-3c68-4e04-85a2-0a2b9e72354d" (UID: "584e1c18-3c68-4e04-85a2-0a2b9e72354d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:21.301079 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.301055    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn" pod "584e1c18-3c68-4e04-85a2-0a2b9e72354d" (UID: "584e1c18-3c68-4e04-85a2-0a2b9e72354d"). InnerVolumeSpecName "kube-api-access-2c8dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:21.400556 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.400518    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c8dn\" (UniqueName: \"kubernetes.io/projected/584e1c18-3c68-4e04-85a2-0a2b9e72354d-kube-api-access-2c8dn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:21.400556 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.400555    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/584e1c18-3c68-4e04-85a2-0a2b9e72354d-maas-api-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:21.965734 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.965699    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj" event={"ID":"584e1c18-3c68-4e04-85a2-0a2b9e72354d","Type":"ContainerDied","Data":"fe5f9b65c23ba325eb693f078406cfdc688922ae6363ded93f2d3b720c3dc656"}
Nov 02 22:00:21.966251 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.965739    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"
Nov 02 22:00:21.966251 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:21.965761    4888 scope.go:122] "RemoveContainer" containerID="03fe8a09ec12ed50f53fae22de6f7a3db10175b6e9f7a95690bc3cce3c85e9eb"
Nov 02 22:00:22.123833 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:22.123794    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:22.171341 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:22.171302    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-4d9ea0-29894280-czdkj"]
Nov 02 22:00:22.260278 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:22.260186    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:00:22.601707 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:22.601619    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" path="/var/lib/kubelet/pods/584e1c18-3c68-4e04-85a2-0a2b9e72354d/volumes"
Nov 02 22:00:22.602200 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:22.602178    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" path="/var/lib/kubelet/pods/ad514094-ac08-456c-9b01-0c4f4d6242cd/volumes"
Nov 02 22:00:35.067134 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:35.067099    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 22:00:50.091723 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.091658    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" podUID="36ce2bd1-c012-44f1-85f9-9311fce8249c" containerName="main" containerID="cri-o://5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179" gracePeriod=45
Nov 02 22:00:50.381191 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.381157    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 22:00:50.457407 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457365    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457615 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457428    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\" (UniqueName: \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457615 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457448    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457615 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457599    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457789 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457658    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\" (UniqueName: \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457789 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457721    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\") pod \"36ce2bd1-c012-44f1-85f9-9311fce8249c\" (UID: \"36ce2bd1-c012-44f1-85f9-9311fce8249c\") "
Nov 02 22:00:50.457789 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457723    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:50.457974 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.457933    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:50.458103 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.458070    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:50.458258 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.458239    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:50.458325 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.458263    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:50.458325 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.458278    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:50.459873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.459832    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:00:50.459984 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.459892    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:00:50.459984 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.459938    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh" pod "36ce2bd1-c012-44f1-85f9-9311fce8249c" (UID: "36ce2bd1-c012-44f1-85f9-9311fce8249c"). InnerVolumeSpecName "kube-api-access-dsbhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:50.559841 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.559802    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ce2bd1-c012-44f1-85f9-9311fce8249c-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:50.559841 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.559834    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsbhh\" (UniqueName: \"kubernetes.io/projected/36ce2bd1-c012-44f1-85f9-9311fce8249c-kube-api-access-dsbhh\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:50.559841 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:50.559843    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ce2bd1-c012-44f1-85f9-9311fce8249c-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:00:51.173286 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.173245    4888 generic.go:356] "Generic (PLEG): container finished" podID="36ce2bd1-c012-44f1-85f9-9311fce8249c" containerID="5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179" exitCode=0
Nov 02 22:00:51.173749 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.173294    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"
Nov 02 22:00:51.173749 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.173327    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" event={"ID":"36ce2bd1-c012-44f1-85f9-9311fce8249c","Type":"ContainerDied","Data":"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"}
Nov 02 22:00:51.173749 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.173370    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc" event={"ID":"36ce2bd1-c012-44f1-85f9-9311fce8249c","Type":"ContainerDied","Data":"54a8e881cb548df61b4e9ec4606cac2b99ff4e6959eb45b7a489a45011cdde50"}
Nov 02 22:00:51.173749 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.173388    4888 scope.go:122] "RemoveContainer" containerID="5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"
Nov 02 22:00:51.183319 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.183296    4888 scope.go:122] "RemoveContainer" containerID="5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"
Nov 02 22:00:51.183641 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:51.183618    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179\": container with ID starting with 5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179 not found: ID does not exist" containerID="5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"
Nov 02 22:00:51.183712 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.183653    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179"} err="failed to get container status \"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179\": rpc error: code = NotFound desc = could not find container \"5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179\": container with ID starting with 5f0722002081e1786ffefbdd358f939376c566011329e7bc84686f837d636179 not found: ID does not exist"
Nov 02 22:00:51.368716 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.368685    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 22:00:51.371286 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:51.371258    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-686bbf/test-model-686bbf-kserve-757d9f9dbc-8wkqc"]
Nov 02 22:00:52.261339 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:00:52.261282    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:00:52.601451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:00:52.601363    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36ce2bd1-c012-44f1-85f9-9311fce8249c" path="/var/lib/kubelet/pods/36ce2bd1-c012-44f1-85f9-9311fce8249c/volumes"
Nov 02 22:01:22.261059 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:01:22.261020    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:01:52.269791 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:01:52.269742    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:02:22.260761 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:02:22.260713    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:02:27.687749 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:27.687718    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 22:02:29.904516 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:29.904477    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 22:02:29.959130 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:29.959098    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 22:02:34.905239 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:34.905175    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" podUID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" containerName="payload-processing" containerID="cri-o://3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8" gracePeriod=25
Nov 02 22:02:34.959638 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:34.959596    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing" containerID="cri-o://1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd" gracePeriod=25
Nov 02 22:02:35.187007 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.186976    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 22:02:35.298094 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.298052    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\") pod \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") "
Nov 02 22:02:35.298297 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.298154    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\" (UniqueName: \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\") pod \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\" (UID: \"ff96a539-9533-4745-b9ac-6c0c58a0f49f\") "
Nov 02 22:02:35.298540 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.298511    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume" pod "ff96a539-9533-4745-b9ac-6c0c58a0f49f" (UID: "ff96a539-9533-4745-b9ac-6c0c58a0f49f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:35.300431 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.300401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7" pod "ff96a539-9533-4745-b9ac-6c0c58a0f49f" (UID: "ff96a539-9533-4745-b9ac-6c0c58a0f49f"). InnerVolumeSpecName "kube-api-access-9dkt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:35.399750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.399711    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff96a539-9533-4745-b9ac-6c0c58a0f49f-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:35.399750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.399749    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dkt7\" (UniqueName: \"kubernetes.io/projected/ff96a539-9533-4745-b9ac-6c0c58a0f49f-kube-api-access-9dkt7\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:35.888108 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.888062    4888 generic.go:356] "Generic (PLEG): container finished" podID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" containerID="3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8" exitCode=0
Nov 02 22:02:35.888336 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.888123    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"
Nov 02 22:02:35.888336 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.888139    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" event={"ID":"ff96a539-9533-4745-b9ac-6c0c58a0f49f","Type":"ContainerDied","Data":"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"}
Nov 02 22:02:35.888336 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.888212    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd" event={"ID":"ff96a539-9533-4745-b9ac-6c0c58a0f49f","Type":"ContainerDied","Data":"50017b01a062ffc7051024a7486ced21093445870ced7e8342640773e48879bd"}
Nov 02 22:02:35.888336 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.888232    4888 scope.go:122] "RemoveContainer" containerID="3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"
Nov 02 22:02:35.899638 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.899618    4888 scope.go:122] "RemoveContainer" containerID="3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"
Nov 02 22:02:35.900044 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:02:35.900016    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8\": container with ID starting with 3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8 not found: ID does not exist" containerID="3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"
Nov 02 22:02:35.900122 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:35.900057    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8"} err="failed to get container status \"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8\": rpc error: code = NotFound desc = could not find container \"3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8\": container with ID starting with 3888cb96fc3b723b6c5601c4be644346debe239d2a4f8678eba10aaadb1a8db8 not found: ID does not exist"
Nov 02 22:02:36.048677 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:36.048637    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 22:02:36.054441 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:36.054407    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd"]
Nov 02 22:02:36.600433 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:36.600395    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" path="/var/lib/kubelet/pods/ff96a539-9533-4745-b9ac-6c0c58a0f49f/volumes"
Nov 02 22:02:42.709028 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:42.708983    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" podUID="1a5476b7-97d8-48be-99de-37fd520e0f8c" containerName="main" containerID="cri-o://823a6d69fed7015940e72e7065fcdd8f6239a0c3b46b037f7732e026e884b8a6" gracePeriod=45
Nov 02 22:02:42.935369 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:42.935336    4888 generic.go:356] "Generic (PLEG): container finished" podID="1a5476b7-97d8-48be-99de-37fd520e0f8c" containerID="823a6d69fed7015940e72e7065fcdd8f6239a0c3b46b037f7732e026e884b8a6" exitCode=0
Nov 02 22:02:42.935530 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:42.935416    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" event={"ID":"1a5476b7-97d8-48be-99de-37fd520e0f8c","Type":"ContainerDied","Data":"823a6d69fed7015940e72e7065fcdd8f6239a0c3b46b037f7732e026e884b8a6"}
Nov 02 22:02:42.977160 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:42.977102    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 22:02:43.006185 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006154    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006355 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006204    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\" (UniqueName: \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006355 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006260    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\" (UniqueName: \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006355 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006281    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006521 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006378    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006521 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006466    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\") pod \"1a5476b7-97d8-48be-99de-37fd520e0f8c\" (UID: \"1a5476b7-97d8-48be-99de-37fd520e0f8c\") "
Nov 02 22:02:43.006521 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006472    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:43.006747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006716    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:43.006901 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.006800    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:43.007021 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.007000    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.007085 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.007026    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.007085 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.007042    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.008667 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.008638    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:43.008765 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.008644    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:02:43.008807 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.008759    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf" pod "1a5476b7-97d8-48be-99de-37fd520e0f8c" (UID: "1a5476b7-97d8-48be-99de-37fd520e0f8c"). InnerVolumeSpecName "kube-api-access-dqqwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:43.108279 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.108231    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqqwf\" (UniqueName: \"kubernetes.io/projected/1a5476b7-97d8-48be-99de-37fd520e0f8c-kube-api-access-dqqwf\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.108279 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.108269    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a5476b7-97d8-48be-99de-37fd520e0f8c-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.108279 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.108283    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a5476b7-97d8-48be-99de-37fd520e0f8c-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:43.816036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.815990    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:02:43.942823 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.942778    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m" event={"ID":"1a5476b7-97d8-48be-99de-37fd520e0f8c","Type":"ContainerDied","Data":"d2dcea1a1d995941a199c31ca82d7fe092d21958ce6f9c327c59afb15a23734f"}
Nov 02 22:02:43.942823 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.942803    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"
Nov 02 22:02:43.943091 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:43.942838    4888 scope.go:122] "RemoveContainer" containerID="823a6d69fed7015940e72e7065fcdd8f6239a0c3b46b037f7732e026e884b8a6"
Nov 02 22:02:44.098965 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:44.098835    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 22:02:44.146200 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:44.146156    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c1243c/test-model-c1243c-kserve-67f78f8959-dlw5m"]
Nov 02 22:02:44.315489 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:44.315442    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 22:02:44.600625 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:44.600589    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a5476b7-97d8-48be-99de-37fd520e0f8c" path="/var/lib/kubelet/pods/1a5476b7-97d8-48be-99de-37fd520e0f8c/volumes"
Nov 02 22:02:47.299653 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:47.299612    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 22:02:51.032943 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:51.032878    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:02:51.042419 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:51.042389    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:02:52.260098 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:02:52.260059    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:02:53.815880 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:53.815808    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:02:55.219932 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.219837    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 22:02:55.367319 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.367279    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\" (UniqueName: \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\") pod \"3b752e56-1a34-417a-9847-e99570238998\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") "
Nov 02 22:02:55.367538 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.367381    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\") pod \"3b752e56-1a34-417a-9847-e99570238998\" (UID: \"3b752e56-1a34-417a-9847-e99570238998\") "
Nov 02 22:02:55.367795 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.367768    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume" pod "3b752e56-1a34-417a-9847-e99570238998" (UID: "3b752e56-1a34-417a-9847-e99570238998"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:55.369604 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.369571    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts" pod "3b752e56-1a34-417a-9847-e99570238998" (UID: "3b752e56-1a34-417a-9847-e99570238998"). InnerVolumeSpecName "kube-api-access-zshts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:55.468454 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.468418    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zshts\" (UniqueName: \"kubernetes.io/projected/3b752e56-1a34-417a-9847-e99570238998-kube-api-access-zshts\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:55.468454 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.468449    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b752e56-1a34-417a-9847-e99570238998-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:55.513840 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.513757    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 22:02:55.617530 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:55.617496    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 22:02:56.014955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.014919    4888 generic.go:356] "Generic (PLEG): container finished" podID="3b752e56-1a34-417a-9847-e99570238998" containerID="1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd" exitCode=0
Nov 02 22:02:56.015230 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.014962    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" event={"ID":"3b752e56-1a34-417a-9847-e99570238998","Type":"ContainerDied","Data":"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"}
Nov 02 22:02:56.015230 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.014973    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"
Nov 02 22:02:56.015230 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.015033    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q" event={"ID":"3b752e56-1a34-417a-9847-e99570238998","Type":"ContainerDied","Data":"5351d384771af6693746abf71c13374ac5562ac2feef240650389504f5784795"}
Nov 02 22:02:56.015230 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.015063    4888 scope.go:122] "RemoveContainer" containerID="1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"
Nov 02 22:02:56.034592 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.034559    4888 scope.go:122] "RemoveContainer" containerID="1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"
Nov 02 22:02:56.035206 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:02:56.035181    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd\": container with ID starting with 1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd not found: ID does not exist" containerID="1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"
Nov 02 22:02:56.035331 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.035215    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd"} err="failed to get container status \"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd\": rpc error: code = NotFound desc = could not find container \"1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd\": container with ID starting with 1045ae2e52586cd720483c6dad9b3d2ae6430262277320cc7481f4db911b8acd not found: ID does not exist"
Nov 02 22:02:56.182861 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.182809    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 22:02:56.185813 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.185782    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q"]
Nov 02 22:02:56.601016 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:56.600984    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b752e56-1a34-417a-9847-e99570238998" path="/var/lib/kubelet/pods/3b752e56-1a34-417a-9847-e99570238998/volumes"
Nov 02 22:02:59.343008 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.342939    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" podUID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" containerName="main" containerID="cri-o://469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c" gracePeriod=45
Nov 02 22:02:59.619457 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.619430    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 22:02:59.726181 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726149    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726364 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726209    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\" (UniqueName: \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726364 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726235    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726489 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726405    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726489 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726444    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:59.726595 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726529    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:59.726595 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726539    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726595 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726582    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\") pod \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\" (UID: \"ea1b6e79-6644-41c6-ba68-f3d77bf356d3\") "
Nov 02 22:02:59.726824 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.726786    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:59.727191 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.727169    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:59.727191 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.727195    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:59.727354 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.727209    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:59.729742 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.729598    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:02:59.729742 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.729637    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:59.729742 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.729697    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx" pod "ea1b6e79-6644-41c6-ba68-f3d77bf356d3" (UID: "ea1b6e79-6644-41c6-ba68-f3d77bf356d3"). InnerVolumeSpecName "kube-api-access-qqfvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:59.828909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.828840    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:59.828909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.828902    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:02:59.828909 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:02:59.828916    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqfvx\" (UniqueName: \"kubernetes.io/projected/ea1b6e79-6644-41c6-ba68-f3d77bf356d3-kube-api-access-qqfvx\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:00.039575 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.039537    4888 generic.go:356] "Generic (PLEG): container finished" podID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" containerID="469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c" exitCode=0
Nov 02 22:03:00.039760 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.039583    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"
Nov 02 22:03:00.039760 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.039625    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" event={"ID":"ea1b6e79-6644-41c6-ba68-f3d77bf356d3","Type":"ContainerDied","Data":"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"}
Nov 02 22:03:00.039760 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.039667    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf" event={"ID":"ea1b6e79-6644-41c6-ba68-f3d77bf356d3","Type":"ContainerDied","Data":"d25625032b3c0e83a5e439d28d1a3eb7a107e2a195422f1347846f23faf2f190"}
Nov 02 22:03:00.039760 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.039687    4888 scope.go:122] "RemoveContainer" containerID="469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"
Nov 02 22:03:00.050466 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.050446    4888 scope.go:122] "RemoveContainer" containerID="469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"
Nov 02 22:03:00.050754 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:00.050734    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c\": container with ID starting with 469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c not found: ID does not exist" containerID="469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"
Nov 02 22:03:00.050837 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.050764    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c"} err="failed to get container status \"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c\": rpc error: code = NotFound desc = could not find container \"469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c\": container with ID starting with 469c87fded79154ca632cabb6a8b3c73cfc58039056f5d2613fdbc8a8f6fb27c not found: ID does not exist"
Nov 02 22:03:00.196803 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.196762    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 22:03:00.240187 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.240154    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-70d237/test-model-70d237-kserve-6c47468c5c-gw6vf"]
Nov 02 22:03:00.514588 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.514540    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" podUID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" containerName="payload-processing" containerID="cri-o://aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867" gracePeriod=25
Nov 02 22:03:00.601212 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.601174    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" path="/var/lib/kubelet/pods/ea1b6e79-6644-41c6-ba68-f3d77bf356d3/volumes"
Nov 02 22:03:00.618447 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.618412    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing" containerID="cri-o://2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3" gracePeriod=25
Nov 02 22:03:00.982870 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:00.982809    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 22:03:01.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.054073    4888 generic.go:356] "Generic (PLEG): container finished" podID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" containerID="aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867" exitCode=0
Nov 02 22:03:01.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.054259    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" event={"ID":"13e03ac2-be04-4d3e-ac44-f319bfe2e789","Type":"ContainerDied","Data":"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"}
Nov 02 22:03:01.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.054349    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv" event={"ID":"13e03ac2-be04-4d3e-ac44-f319bfe2e789","Type":"ContainerDied","Data":"ffed1c6eeacf33dd981a44c6d66c813a6275237b510e112184c0ef99bf1ea4ff"}
Nov 02 22:03:01.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.054387    4888 scope.go:122] "RemoveContainer" containerID="aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"
Nov 02 22:03:01.054873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.054554    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"
Nov 02 22:03:01.071729 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.071609    4888 scope.go:122] "RemoveContainer" containerID="aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"
Nov 02 22:03:01.072145 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:01.072045    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867\": container with ID starting with aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867 not found: ID does not exist" containerID="aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"
Nov 02 22:03:01.072145 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.072084    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867"} err="failed to get container status \"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867\": rpc error: code = NotFound desc = could not find container \"aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867\": container with ID starting with aa88e4ff9b9688ca16c929fa4411092987b7c1a5ce6046622953b885ca8d3867 not found: ID does not exist"
Nov 02 22:03:01.077812 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.077782    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\") pod \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") "
Nov 02 22:03:01.077944 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.077923    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\" (UniqueName: \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\") pod \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\" (UID: \"13e03ac2-be04-4d3e-ac44-f319bfe2e789\") "
Nov 02 22:03:01.078263 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.078232    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume" pod "13e03ac2-be04-4d3e-ac44-f319bfe2e789" (UID: "13e03ac2-be04-4d3e-ac44-f319bfe2e789"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:01.078420 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.078399    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13e03ac2-be04-4d3e-ac44-f319bfe2e789-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:01.084952 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.084917    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl" pod "13e03ac2-be04-4d3e-ac44-f319bfe2e789" (UID: "13e03ac2-be04-4d3e-ac44-f319bfe2e789"). InnerVolumeSpecName "kube-api-access-jsfpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:01.179695 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.179653    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsfpl\" (UniqueName: \"kubernetes.io/projected/13e03ac2-be04-4d3e-ac44-f319bfe2e789-kube-api-access-jsfpl\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:01.469717 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.469623    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 22:03:01.471963 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:01.471935    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv"]
Nov 02 22:03:02.321071 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.321031    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" containerID="cri-o://b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c" gracePeriod=45
Nov 02 22:03:02.402871 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.402817    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": dial tcp 10.132.0.47:8000: connect: connection refused"
Nov 02 22:03:02.596436 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.596408    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 22:03:02.599686 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.599658    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" path="/var/lib/kubelet/pods/13e03ac2-be04-4d3e-ac44-f319bfe2e789/volumes"
Nov 02 22:03:02.697996 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.697965    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\" (UniqueName: \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698203 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698065    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698203 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698126    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698203 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698193    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698379 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698270    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698379 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698334    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\" (UniqueName: \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\") pod \"9b27adf3-f506-4702-92cd-becfd90326a3\" (UID: \"9b27adf3-f506-4702-92cd-becfd90326a3\") "
Nov 02 22:03:02.698483 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698381    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:02.698537 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698475    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:02.698604 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698578    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:02.698811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698776    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:02.698962 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698812    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:02.698962 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.698829    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:02.700353 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.700331    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "kube-api-access-stsps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:02.700557 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.700533    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:02.700893 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.700872    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs" pod "9b27adf3-f506-4702-92cd-becfd90326a3" (UID: "9b27adf3-f506-4702-92cd-becfd90326a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:03:02.799674 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.799635    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b27adf3-f506-4702-92cd-becfd90326a3-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:02.799674 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.799670    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stsps\" (UniqueName: \"kubernetes.io/projected/9b27adf3-f506-4702-92cd-becfd90326a3-kube-api-access-stsps\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:02.799897 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:02.799685    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b27adf3-f506-4702-92cd-becfd90326a3-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:03.068007 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.067971    4888 generic.go:356] "Generic (PLEG): container finished" podID="9b27adf3-f506-4702-92cd-becfd90326a3" containerID="b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c" exitCode=0
Nov 02 22:03:03.068211 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.068028    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"
Nov 02 22:03:03.068211 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.068046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" event={"ID":"9b27adf3-f506-4702-92cd-becfd90326a3","Type":"ContainerDied","Data":"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"}
Nov 02 22:03:03.068211 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.068093    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv" event={"ID":"9b27adf3-f506-4702-92cd-becfd90326a3","Type":"ContainerDied","Data":"07eb349896daf5c91e15a83cdd62b139af7edcb59856903fe9d00c6104f21503"}
Nov 02 22:03:03.068211 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.068110    4888 scope.go:122] "RemoveContainer" containerID="b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"
Nov 02 22:03:03.078035 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.078009    4888 scope.go:122] "RemoveContainer" containerID="b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"
Nov 02 22:03:03.078443 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:03.078346    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c\": container with ID starting with b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c not found: ID does not exist" containerID="b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"
Nov 02 22:03:03.078443 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.078391    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c"} err="failed to get container status \"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c\": rpc error: code = NotFound desc = could not find container \"b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c\": container with ID starting with b1b482789c677d150372adbd17790c0da3065e4f1fe9eee862c29f1f8cfc157c not found: ID does not exist"
Nov 02 22:03:03.224710 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.224678    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 22:03:03.271915 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:03.271877    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-c2dd3b/premium-w1-c2dd3b-kserve-6c68f8746f-4bkjv"]
Nov 02 22:03:04.601117 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:04.601083    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" path="/var/lib/kubelet/pods/9b27adf3-f506-4702-92cd-becfd90326a3/volumes"
Nov 02 22:03:10.464258 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:10.464219    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:03:17.952314 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:17.952267    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 22:03:17.952870 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:17.952566    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy" containerID="cri-o://63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e" gracePeriod=30
Nov 02 22:03:19.446063 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:19.446019    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 22:03:19.446547 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:19.446340    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy" containerID="cri-o://1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44" gracePeriod=30
Nov 02 22:03:20.464031 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.463995    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:03:20.884285 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.884259    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 22:03:20.978584 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.978539    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\") pod \"2b1cc552-3f30-40b9-a564-01fca3fe3440\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") "
Nov 02 22:03:20.978775 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.978630    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\") pod \"2b1cc552-3f30-40b9-a564-01fca3fe3440\" (UID: \"2b1cc552-3f30-40b9-a564-01fca3fe3440\") "
Nov 02 22:03:20.979061 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.979034    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume" pod "2b1cc552-3f30-40b9-a564-01fca3fe3440" (UID: "2b1cc552-3f30-40b9-a564-01fca3fe3440"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:20.980834 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:20.980783    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx" pod "2b1cc552-3f30-40b9-a564-01fca3fe3440" (UID: "2b1cc552-3f30-40b9-a564-01fca3fe3440"). InnerVolumeSpecName "kube-api-access-bjzsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:21.080042 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.080005    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/2b1cc552-3f30-40b9-a564-01fca3fe3440-kube-api-access-bjzsx\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:21.080042 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.080041    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1cc552-3f30-40b9-a564-01fca3fe3440-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:21.240208 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.240108    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerID="2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3" exitCode=0
Nov 02 22:03:21.240208 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.240160    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"
Nov 02 22:03:21.240208 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.240194    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" event={"ID":"2b1cc552-3f30-40b9-a564-01fca3fe3440","Type":"ContainerDied","Data":"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"}
Nov 02 22:03:21.240428 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.240278    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r" event={"ID":"2b1cc552-3f30-40b9-a564-01fca3fe3440","Type":"ContainerDied","Data":"0b210827763ea5a3edc79ca8c532fdb9eddb83c0d11832f6fa13e414395d3a27"}
Nov 02 22:03:21.240428 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.240308    4888 scope.go:122] "RemoveContainer" containerID="2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"
Nov 02 22:03:21.250997 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.250978    4888 scope.go:122] "RemoveContainer" containerID="2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"
Nov 02 22:03:21.251276 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:21.251255    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3\": container with ID starting with 2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3 not found: ID does not exist" containerID="2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"
Nov 02 22:03:21.251327 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.251287    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3"} err="failed to get container status \"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3\": rpc error: code = NotFound desc = could not find container \"2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3\": container with ID starting with 2c6ba45da0f6e60089715f9c8cf2d01d3dffd024fa37adf4dd4bb58f1111bbf3 not found: ID does not exist"
Nov 02 22:03:21.396525 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.396488    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 22:03:21.399087 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:21.399061    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r"]
Nov 02 22:03:22.261541 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:22.261508    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:03:22.600166 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:22.600076    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" path="/var/lib/kubelet/pods/2b1cc552-3f30-40b9-a564-01fca3fe3440/volumes"
Nov 02 22:03:23.232753 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.232691    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 22:03:23.255979 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.255944    4888 generic.go:356] "Generic (PLEG): container finished" podID="851f0db2-7ab7-477a-95e9-310d2516314a" containerID="63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e" exitCode=0
Nov 02 22:03:23.256138 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.255998    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"
Nov 02 22:03:23.256138 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.256025    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerDied","Data":"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"}
Nov 02 22:03:23.256138 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.256083    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db" event={"ID":"851f0db2-7ab7-477a-95e9-310d2516314a","Type":"ContainerDied","Data":"c72740af528e40dad0d08c17c6f71f969d29e0b7d43a71c1eb7e3f05d6c651d2"}
Nov 02 22:03:23.256138 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.256106    4888 scope.go:122] "RemoveContainer" containerID="63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"
Nov 02 22:03:23.265502 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.265483    4888 scope.go:122] "RemoveContainer" containerID="e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"
Nov 02 22:03:23.274494 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.274473    4888 scope.go:122] "RemoveContainer" containerID="63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"
Nov 02 22:03:23.274831 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:23.274809    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e\": container with ID starting with 63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e not found: ID does not exist" containerID="63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"
Nov 02 22:03:23.274913 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.274840    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e"} err="failed to get container status \"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e\": rpc error: code = NotFound desc = could not find container \"63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e\": container with ID starting with 63eae3a52391704268fb6c3f8dc6bc8a0edb9c6569cf2ec54289705e26ac854e not found: ID does not exist"
Nov 02 22:03:23.274913 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.274890    4888 scope.go:122] "RemoveContainer" containerID="e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"
Nov 02 22:03:23.275164 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:23.275148    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9\": container with ID starting with e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9 not found: ID does not exist" containerID="e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"
Nov 02 22:03:23.275228 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.275167    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9"} err="failed to get container status \"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9\": rpc error: code = NotFound desc = could not find container \"e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9\": container with ID starting with e774043d4212304b88fb8a24819249c930f44f56424fab367eb839ac15d7bfb9 not found: ID does not exist"
Nov 02 22:03:23.312257 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312220    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312415 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312275    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312415 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312297    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312415 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312330    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312415 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312368    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312625 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312414    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312625 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312540    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312625 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312592    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:23.312793 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312598    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312793 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312679    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:23.312793 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312694    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\") pod \"851f0db2-7ab7-477a-95e9-310d2516314a\" (UID: \"851f0db2-7ab7-477a-95e9-310d2516314a\") "
Nov 02 22:03:23.312793 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312770    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:23.313029 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.312909    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:23.313029 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313013    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:23.313270 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313252    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.313351 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313275    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.313351 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313290    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.313351 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313303    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/851f0db2-7ab7-477a-95e9-310d2516314a-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.313351 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.313318    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.314654 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.314626    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "kube-api-access-zljts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:23.314957 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.314938    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:23.315222 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.315206    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:03:23.315303 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.315290    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token" pod "851f0db2-7ab7-477a-95e9-310d2516314a" (UID: "851f0db2-7ab7-477a-95e9-310d2516314a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:23.414624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.414581    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zljts\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-kube-api-access-zljts\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.414624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.414618    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/851f0db2-7ab7-477a-95e9-310d2516314a-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.414832 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.414632    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/851f0db2-7ab7-477a-95e9-310d2516314a-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.414832 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.414647    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/851f0db2-7ab7-477a-95e9-310d2516314a-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:23.672777 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.672731    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 22:03:23.676326 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:23.676300    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db"]
Nov 02 22:03:24.602092 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.602056    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" path="/var/lib/kubelet/pods/851f0db2-7ab7-477a-95e9-310d2516314a/volumes"
Nov 02 22:03:24.713067 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.712996    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 22:03:24.832197 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832154    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832197 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832205    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832470 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832235    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832470 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832254    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832470 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832313    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832470 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832360    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832470 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832432    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832472    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832500    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\") pod \"accd6e5a-552b-4e7a-a121-5ae12c683075\" (UID: \"accd6e5a-552b-4e7a-a121-5ae12c683075\") "
Nov 02 22:03:24.832748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832616    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:24.832748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832651    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:24.833023 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832880    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:24.833023 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.833004    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:24.833155 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.832782    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:24.833155 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.833094    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.833155 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.833114    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accd6e5a-552b-4e7a-a121-5ae12c683075-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.833155 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.833128    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.834714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.834686    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:03:24.834833 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.834766    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:24.834833 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.834771    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "kube-api-access-qng7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:24.834833 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.834785    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token" pod "accd6e5a-552b-4e7a-a121-5ae12c683075" (UID: "accd6e5a-552b-4e7a-a121-5ae12c683075"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:24.934313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934276    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.934313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934305    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.934313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934314    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.934313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934325    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accd6e5a-552b-4e7a-a121-5ae12c683075-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.934599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934334    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qng7r\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-kube-api-access-qng7r\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:24.934599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:24.934343    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accd6e5a-552b-4e7a-a121-5ae12c683075-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:25.269979 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.269941    4888 generic.go:356] "Generic (PLEG): container finished" podID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerID="1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44" exitCode=0
Nov 02 22:03:25.270174 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.269993    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"
Nov 02 22:03:25.270174 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.270037    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerDied","Data":"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"}
Nov 02 22:03:25.270174 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.270100    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v" event={"ID":"accd6e5a-552b-4e7a-a121-5ae12c683075","Type":"ContainerDied","Data":"d2549a459193e90faf262c24e0d7be3bcdb4c3e1f1ed48934232428c2abcf4a1"}
Nov 02 22:03:25.270174 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.270130    4888 scope.go:122] "RemoveContainer" containerID="1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"
Nov 02 22:03:25.281191 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.281160    4888 scope.go:122] "RemoveContainer" containerID="8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"
Nov 02 22:03:25.291200 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.291178    4888 scope.go:122] "RemoveContainer" containerID="1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"
Nov 02 22:03:25.291513 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:25.291482    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44\": container with ID starting with 1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44 not found: ID does not exist" containerID="1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"
Nov 02 22:03:25.291553 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.291523    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44"} err="failed to get container status \"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44\": rpc error: code = NotFound desc = could not find container \"1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44\": container with ID starting with 1bee6fee7116b6e97cbdfb639abeded26a66bbe6e3425f806a8a6443baa5aa44 not found: ID does not exist"
Nov 02 22:03:25.291553 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.291544    4888 scope.go:122] "RemoveContainer" containerID="8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"
Nov 02 22:03:25.291821 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:25.291802    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9\": container with ID starting with 8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9 not found: ID does not exist" containerID="8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"
Nov 02 22:03:25.291880 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.291828    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9"} err="failed to get container status \"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9\": rpc error: code = NotFound desc = could not find container \"8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9\": container with ID starting with 8e0aade3390ec2aa61ef60bde0c03223d962192cc55e9cf1c320edee7b73f4b9 not found: ID does not exist"
Nov 02 22:03:25.426557 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.426516    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 22:03:25.430283 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:25.430253    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v"]
Nov 02 22:03:26.599309 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:26.599270    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" path="/var/lib/kubelet/pods/accd6e5a-552b-4e7a-a121-5ae12c683075/volumes"
Nov 02 22:03:30.098837 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:30.098798    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 22:03:34.587435 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:34.587394    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 22:03:34.689369 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:34.689329    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 22:03:35.099225 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.099168    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" podUID="2839fbe0-7269-435f-90cf-23f5eef50384" containerName="payload-processing" containerID="cri-o://2aecf43af3e294bbcd35ea9aa94db74cfb8bf6a833189a08f59f294332d398f0" gracePeriod=25
Nov 02 22:03:35.332394 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.332358    4888 generic.go:356] "Generic (PLEG): container finished" podID="2839fbe0-7269-435f-90cf-23f5eef50384" containerID="2aecf43af3e294bbcd35ea9aa94db74cfb8bf6a833189a08f59f294332d398f0" exitCode=0
Nov 02 22:03:35.332596 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.332429    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" event={"ID":"2839fbe0-7269-435f-90cf-23f5eef50384","Type":"ContainerDied","Data":"2aecf43af3e294bbcd35ea9aa94db74cfb8bf6a833189a08f59f294332d398f0"}
Nov 02 22:03:35.363701 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.363639    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 22:03:35.460694 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.460662    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\") pod \"2839fbe0-7269-435f-90cf-23f5eef50384\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") "
Nov 02 22:03:35.460919 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.460835    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\" (UniqueName: \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\") pod \"2839fbe0-7269-435f-90cf-23f5eef50384\" (UID: \"2839fbe0-7269-435f-90cf-23f5eef50384\") "
Nov 02 22:03:35.461136 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.461111    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume" pod "2839fbe0-7269-435f-90cf-23f5eef50384" (UID: "2839fbe0-7269-435f-90cf-23f5eef50384"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:35.463162 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.463132    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9" pod "2839fbe0-7269-435f-90cf-23f5eef50384" (UID: "2839fbe0-7269-435f-90cf-23f5eef50384"). InnerVolumeSpecName "kube-api-access-kb5z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:35.561806 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.561770    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb5z9\" (UniqueName: \"kubernetes.io/projected/2839fbe0-7269-435f-90cf-23f5eef50384-kube-api-access-kb5z9\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:35.561806 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:35.561804    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2839fbe0-7269-435f-90cf-23f5eef50384-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:36.340119 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.340084    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"
Nov 02 22:03:36.340619 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.340080    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n" event={"ID":"2839fbe0-7269-435f-90cf-23f5eef50384","Type":"ContainerDied","Data":"6c7b0bd159b8ef78185447e2a9590b97e12a4233564fa5a43b0b97d1b320c189"}
Nov 02 22:03:36.340619 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.340212    4888 scope.go:122] "RemoveContainer" containerID="2aecf43af3e294bbcd35ea9aa94db74cfb8bf6a833189a08f59f294332d398f0"
Nov 02 22:03:36.498105 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.498069    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 22:03:36.578900 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.572285    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n"]
Nov 02 22:03:36.601037 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:36.600952    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2839fbe0-7269-435f-90cf-23f5eef50384" path="/var/lib/kubelet/pods/2839fbe0-7269-435f-90cf-23f5eef50384/volumes"
Nov 02 22:03:39.587959 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:39.587876    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" podUID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" containerName="payload-processing" containerID="cri-o://61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7" gracePeriod=25
Nov 02 22:03:39.690081 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:39.689987    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" podUID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" containerName="payload-pre-processing" containerID="cri-o://36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb" gracePeriod=25
Nov 02 22:03:39.860162 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:39.860123    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 22:03:39.970885 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:39.970843    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 22:03:40.010228 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.010194    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\") pod \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") "
Nov 02 22:03:40.010435 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.010370    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\" (UniqueName: \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\") pod \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\" (UID: \"1921f4a1-4fcf-4a87-9600-de8bd25ce0be\") "
Nov 02 22:03:40.010691 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.010653    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume" pod "1921f4a1-4fcf-4a87-9600-de8bd25ce0be" (UID: "1921f4a1-4fcf-4a87-9600-de8bd25ce0be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:40.012637 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.012612    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j" pod "1921f4a1-4fcf-4a87-9600-de8bd25ce0be" (UID: "1921f4a1-4fcf-4a87-9600-de8bd25ce0be"). InnerVolumeSpecName "kube-api-access-fxl8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:40.111206 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111124    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\") pod \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") "
Nov 02 22:03:40.111379 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111215    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\" (UniqueName: \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\") pod \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\" (UID: \"7a2fc1de-91b7-4350-ac03-c63922a3af6c\") "
Nov 02 22:03:40.111500 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111474    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume" pod "7a2fc1de-91b7-4350-ac03-c63922a3af6c" (UID: "7a2fc1de-91b7-4350-ac03-c63922a3af6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:40.111630 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111607    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxl8j\" (UniqueName: \"kubernetes.io/projected/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-kube-api-access-fxl8j\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:40.111678 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111637    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2fc1de-91b7-4350-ac03-c63922a3af6c-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:40.111678 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.111653    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1921f4a1-4fcf-4a87-9600-de8bd25ce0be-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:40.113403 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.113379    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz" pod "7a2fc1de-91b7-4350-ac03-c63922a3af6c" (UID: "7a2fc1de-91b7-4350-ac03-c63922a3af6c"). InnerVolumeSpecName "kube-api-access-28xbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:40.212904 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.212864    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28xbz\" (UniqueName: \"kubernetes.io/projected/7a2fc1de-91b7-4350-ac03-c63922a3af6c-kube-api-access-28xbz\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:40.366048 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.365960    4888 generic.go:356] "Generic (PLEG): container finished" podID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" containerID="36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb" exitCode=0
Nov 02 22:03:40.366048 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.366015    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"
Nov 02 22:03:40.366265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.366035    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" event={"ID":"7a2fc1de-91b7-4350-ac03-c63922a3af6c","Type":"ContainerDied","Data":"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"}
Nov 02 22:03:40.366265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.366099    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh" event={"ID":"7a2fc1de-91b7-4350-ac03-c63922a3af6c","Type":"ContainerDied","Data":"f41d43c0bb29bb1d81c59325892b7831926890986b4821459b7383c8c0127b9b"}
Nov 02 22:03:40.366265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.366131    4888 scope.go:122] "RemoveContainer" containerID="36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"
Nov 02 22:03:40.367504 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.367482    4888 generic.go:356] "Generic (PLEG): container finished" podID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" containerID="61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7" exitCode=0
Nov 02 22:03:40.367596 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.367529    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"
Nov 02 22:03:40.367596 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.367537    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" event={"ID":"1921f4a1-4fcf-4a87-9600-de8bd25ce0be","Type":"ContainerDied","Data":"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"}
Nov 02 22:03:40.367596 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.367567    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq" event={"ID":"1921f4a1-4fcf-4a87-9600-de8bd25ce0be","Type":"ContainerDied","Data":"72fc30cb51882bab97ca38a40a6fed2ed414275086aeaf91f87a23ac0d049fc6"}
Nov 02 22:03:40.376095 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.376075    4888 scope.go:122] "RemoveContainer" containerID="36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"
Nov 02 22:03:40.376364 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:40.376343    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb\": container with ID starting with 36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb not found: ID does not exist" containerID="36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"
Nov 02 22:03:40.376442 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.376372    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb"} err="failed to get container status \"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb\": rpc error: code = NotFound desc = could not find container \"36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb\": container with ID starting with 36062a71218a872d1b44064d96e4e84c275e4743e959c80037db08eb6ca8c4eb not found: ID does not exist"
Nov 02 22:03:40.376442 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.376391    4888 scope.go:122] "RemoveContainer" containerID="61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"
Nov 02 22:03:40.385283 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.385266    4888 scope.go:122] "RemoveContainer" containerID="61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"
Nov 02 22:03:40.385576 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:40.385557    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7\": container with ID starting with 61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7 not found: ID does not exist" containerID="61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"
Nov 02 22:03:40.385642 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.385583    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7"} err="failed to get container status \"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7\": rpc error: code = NotFound desc = could not find container \"61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7\": container with ID starting with 61025e507e6aae5db33fb81317d62475da60c63222ac7a4b44471c8b28e8b7f7 not found: ID does not exist"
Nov 02 22:03:40.520524 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.520491    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 22:03:40.568328 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.568288    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh"]
Nov 02 22:03:40.599530 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.599494    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" path="/var/lib/kubelet/pods/7a2fc1de-91b7-4350-ac03-c63922a3af6c/volumes"
Nov 02 22:03:40.671661 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.671631    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 22:03:40.673623 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:40.673597    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq"]
Nov 02 22:03:42.599710 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:42.599669    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" path="/var/lib/kubelet/pods/1921f4a1-4fcf-4a87-9600-de8bd25ce0be/volumes"
Nov 02 22:03:44.295119 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:44.295080    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 22:03:44.295573 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:44.295340    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy" containerID="cri-o://bcbe74cc0205c4b43c69c3372b7eab7331a7e57c0f0e14ffd7839955474bedcf" gracePeriod=30
Nov 02 22:03:49.436387 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.436352    4888 generic.go:356] "Generic (PLEG): container finished" podID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerID="bcbe74cc0205c4b43c69c3372b7eab7331a7e57c0f0e14ffd7839955474bedcf" exitCode=0
Nov 02 22:03:49.436776 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.436427    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerDied","Data":"bcbe74cc0205c4b43c69c3372b7eab7331a7e57c0f0e14ffd7839955474bedcf"}
Nov 02 22:03:49.436776 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.436489    4888 scope.go:122] "RemoveContainer" containerID="e292c25f64d24d362907cac5cf72b1458754a6a46300e380fce5eebdeede4f71"
Nov 02 22:03:49.558526 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.558499    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 22:03:49.636949 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.636837    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.636949 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.636918    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.636954    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.636978    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637018    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637065    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637128    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637165    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637212    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\") pod \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\" (UID: \"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66\") "
Nov 02 22:03:49.637282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637262    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:49.637703 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637447    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:03:49.637703 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637519    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:49.637703 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637650    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:49.637920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637860    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.637920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637886    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.637920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637901    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.637920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.637917    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.638236 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.638211    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:49.639365 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.639339    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:49.639365 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.639353    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:03:49.639510 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.639374    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:03:49.639510 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.639421    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447" pod "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" (UID: "2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66"). InnerVolumeSpecName "kube-api-access-9v447". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:03:49.739291 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.739256    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.739291 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.739290    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v447\" (UniqueName: \"kubernetes.io/projected/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-kube-api-access-9v447\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.739291 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.739300    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.739581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.739309    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:49.739581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:49.739317    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:03:50.442862 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:50.442828    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"
Nov 02 22:03:50.443355 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:50.442867    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9" event={"ID":"2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66","Type":"ContainerDied","Data":"560eeb2c4ed3c30983386cfe64f786585d76f93fec4a4f37fd0e6feafb0c17c7"}
Nov 02 22:03:50.443355 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:50.442929    4888 scope.go:122] "RemoveContainer" containerID="bcbe74cc0205c4b43c69c3372b7eab7331a7e57c0f0e14ffd7839955474bedcf"
Nov 02 22:03:50.648716 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:50.648679    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 22:03:50.653718 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:50.653685    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9"]
Nov 02 22:03:52.259713 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:03:52.259673    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:03:52.601003 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:03:52.600906    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" path="/var/lib/kubelet/pods/2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66/volumes"
Nov 02 22:04:09.569262 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:09.569225    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 22:04:09.569913 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:09.569500    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy" containerID="cri-o://55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b" gracePeriod=30
Nov 02 22:04:14.838331 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.838255    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 22:04:14.934719 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934684    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.934955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934781    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.934955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934813    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.934955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934833    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.934955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934893    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.934955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.934932    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.935253 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935042    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.935253 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935070    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.935253 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935081    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:14.935253 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935119    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\") pod \"f8f964ac-042b-4dfc-b166-2776d59f3673\" (UID: \"f8f964ac-042b-4dfc-b166-2776d59f3673\") "
Nov 02 22:04:14.935253 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935217    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:14.935536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935371    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:04:14.935536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935501    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:14.935654 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935550    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:14.935654 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935569    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8f964ac-042b-4dfc-b166-2776d59f3673-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:14.935654 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935584    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:14.935822 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.935665    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:14.937215 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.937192    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:04:14.937373 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.937351    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "kube-api-access-n2hvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:14.937451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.937375    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:14.937685 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.937537    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token" pod "f8f964ac-042b-4dfc-b166-2776d59f3673" (UID: "f8f964ac-042b-4dfc-b166-2776d59f3673"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:14.994364 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:14.994327    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 22:04:15.036317 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036269    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.036317 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036301    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.036317 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036314    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.036317 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036326    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hvm\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-kube-api-access-n2hvm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.036653 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036337    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.036653 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.036347    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8f964ac-042b-4dfc-b166-2776d59f3673-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:15.045006 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.044971    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 22:04:15.586803 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.586759    4888 generic.go:356] "Generic (PLEG): container finished" podID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerID="55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b" exitCode=0
Nov 02 22:04:15.587036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.586828    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"
Nov 02 22:04:15.587036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.586841    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerDied","Data":"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"}
Nov 02 22:04:15.587036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.586935    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2" event={"ID":"f8f964ac-042b-4dfc-b166-2776d59f3673","Type":"ContainerDied","Data":"36cfc93f7315951a0040981be2eda01664d1a7ded1872cc51c3df41130d0465c"}
Nov 02 22:04:15.587036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.586967    4888 scope.go:122] "RemoveContainer" containerID="55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"
Nov 02 22:04:15.597697 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.597672    4888 scope.go:122] "RemoveContainer" containerID="7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"
Nov 02 22:04:15.608552 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.608522    4888 scope.go:122] "RemoveContainer" containerID="55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"
Nov 02 22:04:15.608926 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:15.608905    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b\": container with ID starting with 55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b not found: ID does not exist" containerID="55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"
Nov 02 22:04:15.609026 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.608935    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b"} err="failed to get container status \"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b\": rpc error: code = NotFound desc = could not find container \"55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b\": container with ID starting with 55b89585d5c46b8a434d5ee049d4a8782dbbba8a0664a8fd0b8e4eff52a41e6b not found: ID does not exist"
Nov 02 22:04:15.609026 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.608952    4888 scope.go:122] "RemoveContainer" containerID="7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"
Nov 02 22:04:15.609265 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:15.609242    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d\": container with ID starting with 7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d not found: ID does not exist" containerID="7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"
Nov 02 22:04:15.609312 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.609274    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d"} err="failed to get container status \"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d\": rpc error: code = NotFound desc = could not find container \"7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d\": container with ID starting with 7283e9049e012b6c2392d583f19bd62d26a4147ba94e0c0c70ccdbc088c19c0d not found: ID does not exist"
Nov 02 22:04:15.744700 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.744665    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 22:04:15.748007 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:15.747980    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2"]
Nov 02 22:04:16.600031 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:16.599993    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" path="/var/lib/kubelet/pods/f8f964ac-042b-4dfc-b166-2776d59f3673/volumes"
Nov 02 22:04:20.727043 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:20.726999    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 22:04:20.727465 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:20.727279    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" containerID="cri-o://d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852" gracePeriod=30
Nov 02 22:04:22.262126 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:22.262087    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:04:23.432273 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:23.432232    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:04:23.432670 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:23.432290    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:04:26.023638 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.023605    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 22:04:26.166108 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166073    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166125    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166148    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166197    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166222    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166273    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166291    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166701 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166339    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166701 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166396    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\") pod \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\" (UID: \"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf\") "
Nov 02 22:04:26.166701 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166466    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:26.166891 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166705    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.166968 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166949    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:26.167024 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.166974    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:26.167124 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.167064    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:26.167287 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.167238    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:04:26.168612 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.168535    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:04:26.168612 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.168599    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "kube-api-access-lnnlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:26.168960 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.168765    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:26.168960 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.168935    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy" pod "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" (UID: "5b23d49a-ff76-4d09-9b0d-ea60fbf037cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:26.237466 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.237423    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267711    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnnlw\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-kube-api-access-lnnlw\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267748    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267758    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267768    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267777    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267786    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267794    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.267798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.267802    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:26.656126 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.656089    4888 generic.go:356] "Generic (PLEG): container finished" podID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerID="d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852" exitCode=0
Nov 02 22:04:26.656335 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.656140    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"
Nov 02 22:04:26.656335 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.656182    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerDied","Data":"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"}
Nov 02 22:04:26.656335 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.656248    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg" event={"ID":"5b23d49a-ff76-4d09-9b0d-ea60fbf037cf","Type":"ContainerDied","Data":"fde1ec28dee1269ec27cac34e3aa3d6d22c570764359fc7707eb73dae876b431"}
Nov 02 22:04:26.656335 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.656280    4888 scope.go:122] "RemoveContainer" containerID="d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"
Nov 02 22:04:26.667549 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.667522    4888 scope.go:122] "RemoveContainer" containerID="34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"
Nov 02 22:04:26.677176 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.677149    4888 scope.go:122] "RemoveContainer" containerID="d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"
Nov 02 22:04:26.677529 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:26.677504    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852\": container with ID starting with d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852 not found: ID does not exist" containerID="d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"
Nov 02 22:04:26.677635 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.677545    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852"} err="failed to get container status \"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852\": rpc error: code = NotFound desc = could not find container \"d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852\": container with ID starting with d5e8ae26de8c1f99bf8823833937f86bf01a1b69359b6fb7ac5b0461c81ed852 not found: ID does not exist"
Nov 02 22:04:26.677635 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.677573    4888 scope.go:122] "RemoveContainer" containerID="34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"
Nov 02 22:04:26.677872 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:26.677838    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff\": container with ID starting with 34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff not found: ID does not exist" containerID="34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"
Nov 02 22:04:26.677936 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.677882    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff"} err="failed to get container status \"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff\": rpc error: code = NotFound desc = could not find container \"34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff\": container with ID starting with 34c5a3e79423901e674a32fc325c817392ae0cb7882fdb0f968bf333a1f521ff not found: ID does not exist"
Nov 02 22:04:26.812959 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.812912    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 22:04:26.815095 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:26.815064    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg"]
Nov 02 22:04:28.609538 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:28.609498    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" path="/var/lib/kubelet/pods/5b23d49a-ff76-4d09-9b0d-ea60fbf037cf/volumes"
Nov 02 22:04:30.015718 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.015623    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" containerID="cri-o://adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630" gracePeriod=45
Nov 02 22:04:30.066597 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.066551    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" containerID="cri-o://30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff" gracePeriod=45
Nov 02 22:04:30.070496 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.070461    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": EOF"
Nov 02 22:04:30.324254 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.324222    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 22:04:30.356388 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.356361    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 22:04:30.419363 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419331    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419564 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419371    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\" (UniqueName: \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419564 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419397    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419564 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419509    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419728 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419583    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\" (UniqueName: \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419728 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419633    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\") pod \"389b1624-08e6-4dc3-bff7-397abbea06f7\" (UID: \"389b1624-08e6-4dc3-bff7-397abbea06f7\") "
Nov 02 22:04:30.419834 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419705    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.419834 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419773    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.419964 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.419947    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.420060 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.420040    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.420060 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.420056    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.420060 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.420065    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.421983 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.421954    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "kube-api-access-kspdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:30.421983 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.421967    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.422130 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.422041    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs" pod "389b1624-08e6-4dc3-bff7-397abbea06f7" (UID: "389b1624-08e6-4dc3-bff7-397abbea06f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:04:30.520746 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.520703    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\" (UniqueName: \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.520968 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.520810    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.520968 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.520919    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.520968 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.520957    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.521105 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521002    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.521105 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521083    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\") pod \"a9990587-21c7-4653-ba57-46a3be01d996\" (UID: \"a9990587-21c7-4653-ba57-46a3be01d996\") "
Nov 02 22:04:30.521275 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521227    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.521411 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521265    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.521411 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521384    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.521526 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521504    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.521581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521530    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.521581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521548    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.521581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521563    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kspdf\" (UniqueName: \"kubernetes.io/projected/389b1624-08e6-4dc3-bff7-397abbea06f7-kube-api-access-kspdf\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.521581 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521578    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b1624-08e6-4dc3-bff7-397abbea06f7-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.521737 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.521593    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b1624-08e6-4dc3-bff7-397abbea06f7-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.523134 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.523103    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:30.523265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.523154    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:04:30.523265 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.523213    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8" pod "a9990587-21c7-4653-ba57-46a3be01d996" (UID: "a9990587-21c7-4653-ba57-46a3be01d996"). InnerVolumeSpecName "kube-api-access-4hnh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:30.623101 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.623063    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9990587-21c7-4653-ba57-46a3be01d996-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.623101 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.623094    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9990587-21c7-4653-ba57-46a3be01d996-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.623101 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.623106    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hnh8\" (UniqueName: \"kubernetes.io/projected/a9990587-21c7-4653-ba57-46a3be01d996-kube-api-access-4hnh8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:30.681292 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.681252    4888 generic.go:356] "Generic (PLEG): container finished" podID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerID="adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630" exitCode=0
Nov 02 22:04:30.681505 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.681308    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"
Nov 02 22:04:30.681505 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.681338    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" event={"ID":"389b1624-08e6-4dc3-bff7-397abbea06f7","Type":"ContainerDied","Data":"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"}
Nov 02 22:04:30.681505 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.681387    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" event={"ID":"389b1624-08e6-4dc3-bff7-397abbea06f7","Type":"ContainerDied","Data":"ffe74e5e4c6a12f4c5e5d86ace1d59054adaf6b8e581e336e389742089bedef2"}
Nov 02 22:04:30.681505 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.681412    4888 scope.go:122] "RemoveContainer" containerID="adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"
Nov 02 22:04:30.682758 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.682731    4888 generic.go:356] "Generic (PLEG): container finished" podID="a9990587-21c7-4653-ba57-46a3be01d996" containerID="30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff" exitCode=0
Nov 02 22:04:30.682914 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.682779    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"
Nov 02 22:04:30.682914 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.682817    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" event={"ID":"a9990587-21c7-4653-ba57-46a3be01d996","Type":"ContainerDied","Data":"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"}
Nov 02 22:04:30.682914 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.682878    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6" event={"ID":"a9990587-21c7-4653-ba57-46a3be01d996","Type":"ContainerDied","Data":"741f7d3f9e413977b74787d4dd72d1a0140c4c1779e93b5304a27a11ac7fd51e"}
Nov 02 22:04:30.690583 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.690554    4888 scope.go:122] "RemoveContainer" containerID="adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"
Nov 02 22:04:30.690928 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:30.690909    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630\": container with ID starting with adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630 not found: ID does not exist" containerID="adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"
Nov 02 22:04:30.690974 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.690938    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630"} err="failed to get container status \"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630\": rpc error: code = NotFound desc = could not find container \"adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630\": container with ID starting with adbd886ea2f93ed351e78ec9ac0bbc92aea922b74401c38087058ebb3eafb630 not found: ID does not exist"
Nov 02 22:04:30.690974 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.690961    4888 scope.go:122] "RemoveContainer" containerID="30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"
Nov 02 22:04:30.699324 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.699300    4888 scope.go:122] "RemoveContainer" containerID="30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"
Nov 02 22:04:30.699660 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:30.699638    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff\": container with ID starting with 30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff not found: ID does not exist" containerID="30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"
Nov 02 22:04:30.699734 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.699673    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff"} err="failed to get container status \"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff\": rpc error: code = NotFound desc = could not find container \"30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff\": container with ID starting with 30ef5de2ac57b5838f1c67fc4dbc243350b332b436e8f1615b7567ac219d33ff not found: ID does not exist"
Nov 02 22:04:30.835869 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.835810    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 22:04:30.839831 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.839799    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/simulator-w3-9a6cf2-kserve-5d89d564b9-txgm6"]
Nov 02 22:04:30.943676 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.943642    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 22:04:30.945571 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:30.945544    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs"]
Nov 02 22:04:31.319911 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:31.319774    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9a6cf2/premium-w3-9a6cf2-kserve-5464db59d4-54pjs" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Nov 02 22:04:32.607457 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:32.607419    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" path="/var/lib/kubelet/pods/389b1624-08e6-4dc3-bff7-397abbea06f7/volumes"
Nov 02 22:04:32.608805 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:32.608779    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9990587-21c7-4653-ba57-46a3be01d996" path="/var/lib/kubelet/pods/a9990587-21c7-4653-ba57-46a3be01d996/volumes"
Nov 02 22:04:37.931995 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:37.931956    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 22:04:37.979300 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:37.979263    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 22:04:39.525497 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:39.525461    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 22:04:39.525961 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:39.525637    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy" containerID="cri-o://48df0f4a6a9151d6a6a93498a78272eb8dc0ca1678033a73b6b66854efd6edfd" gracePeriod=30
Nov 02 22:04:41.261575 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.261512    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main" containerID="cri-o://b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457" gracePeriod=45
Nov 02 22:04:41.531118 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.531090    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 22:04:41.539441 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539415    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539465    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539527    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\" (UniqueName: \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539577    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539596    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539881 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539633    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\") pod \"5d894546-4031-4d92-80eb-7d5f60fb59c1\" (UID: \"5d894546-4031-4d92-80eb-7d5f60fb59c1\") "
Nov 02 22:04:41.539881 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539648    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:41.540003 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539931    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:41.540003 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539945    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:41.540003 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.539940    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.542038 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.542001    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "kube-api-access-5cpq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:41.542038 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.542024    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:41.542234 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.542147    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs" pod "5d894546-4031-4d92-80eb-7d5f60fb59c1" (UID: "5d894546-4031-4d92-80eb-7d5f60fb59c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:04:41.641433 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.641396    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.641433 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.641433    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d894546-4031-4d92-80eb-7d5f60fb59c1-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.641689 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.641448    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cpq4\" (UniqueName: \"kubernetes.io/projected/5d894546-4031-4d92-80eb-7d5f60fb59c1-kube-api-access-5cpq4\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.641689 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.641462    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.641689 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.641475    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d894546-4031-4d92-80eb-7d5f60fb59c1-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:41.744517 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.744469    4888 generic.go:356] "Generic (PLEG): container finished" podID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerID="b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457" exitCode=0
Nov 02 22:04:41.744708 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.744532    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"
Nov 02 22:04:41.744708 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.744542    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" event={"ID":"5d894546-4031-4d92-80eb-7d5f60fb59c1","Type":"ContainerDied","Data":"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"}
Nov 02 22:04:41.744708 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.744596    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j" event={"ID":"5d894546-4031-4d92-80eb-7d5f60fb59c1","Type":"ContainerDied","Data":"bdf78b8e8860f252d8303299445151e563b0b3e9f5187721a42a0dd52b412787"}
Nov 02 22:04:41.744708 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.744617    4888 scope.go:122] "RemoveContainer" containerID="b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"
Nov 02 22:04:41.754074 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.753960    4888 scope.go:122] "RemoveContainer" containerID="b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"
Nov 02 22:04:41.754324 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:41.754301    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457\": container with ID starting with b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457 not found: ID does not exist" containerID="b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"
Nov 02 22:04:41.754402 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.754333    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457"} err="failed to get container status \"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457\": rpc error: code = NotFound desc = could not find container \"b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457\": container with ID starting with b545857604ddc396547761835dec352bc0d7209b471d10a4a74844c5b464c457 not found: ID does not exist"
Nov 02 22:04:41.901784 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.901689    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 22:04:41.903614 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:41.903587    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-4d9ea0/simulator-w2-4d9ea0-kserve-589987c764-fjm4j"]
Nov 02 22:04:42.600623 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:42.600582    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" path="/var/lib/kubelet/pods/5d894546-4031-4d92-80eb-7d5f60fb59c1/volumes"
Nov 02 22:04:44.764451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.764420    4888 generic.go:356] "Generic (PLEG): container finished" podID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerID="48df0f4a6a9151d6a6a93498a78272eb8dc0ca1678033a73b6b66854efd6edfd" exitCode=0
Nov 02 22:04:44.764840 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.764493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" event={"ID":"a3e4da4c-9597-4076-9c20-6556a1c6d009","Type":"ContainerDied","Data":"48df0f4a6a9151d6a6a93498a78272eb8dc0ca1678033a73b6b66854efd6edfd"}
Nov 02 22:04:44.789258 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.789234    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 22:04:44.875499 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875415    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875499 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875505    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875540    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875581    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875630    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875714 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875673    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875997 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875728    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.875997 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875767    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:44.875997 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875778    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\") pod \"a3e4da4c-9597-4076-9c20-6556a1c6d009\" (UID: \"a3e4da4c-9597-4076-9c20-6556a1c6d009\") "
Nov 02 22:04:44.876134 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876021    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:44.876134 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876037    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:44.876134 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.875951    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:04:44.876278 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876252    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.876278 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876274    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.876399 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876289    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.876399 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876304    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3e4da4c-9597-4076-9c20-6556a1c6d009-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.876399 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.876337    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:44.878178 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.878149    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:44.878295 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.878173    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:04:44.878961 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.878931    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "kube-api-access-tgs69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:44.878961 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.878950    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy" pod "a3e4da4c-9597-4076-9c20-6556a1c6d009" (UID: "a3e4da4c-9597-4076-9c20-6556a1c6d009"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:44.976927 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.976889    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgs69\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-kube-api-access-tgs69\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.976927 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.976926    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.977143 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.976941    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.977143 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.976951    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:44.977143 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:44.976961    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3e4da4c-9597-4076-9c20-6556a1c6d009-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:45.772272 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:45.772219    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz" event={"ID":"a3e4da4c-9597-4076-9c20-6556a1c6d009","Type":"ContainerDied","Data":"a92d09a0d97ba319f505c41a1613479f6a3d740bcb0d97cb3bd003942c39eaca"}
Nov 02 22:04:45.772757 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:45.772393    4888 scope.go:122] "RemoveContainer" containerID="48df0f4a6a9151d6a6a93498a78272eb8dc0ca1678033a73b6b66854efd6edfd"
Nov 02 22:04:45.772757 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:45.772636    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"
Nov 02 22:04:45.976607 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:45.976575    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 22:04:46.024498 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:46.024414    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz"]
Nov 02 22:04:46.607623 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:46.607230    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" path="/var/lib/kubelet/pods/a3e4da4c-9597-4076-9c20-6556a1c6d009/volumes"
Nov 02 22:04:52.269877 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:52.269787    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:04:52.953135 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:52.953071    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main" containerID="cri-o://ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30" gracePeriod=45
Nov 02 22:04:52.999112 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:52.999051    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main" containerID="cri-o://e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02" gracePeriod=45
Nov 02 22:04:53.256165 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.256140    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 22:04:53.279687 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.279654    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 22:04:53.367554 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367523    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.367747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367564    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.367747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367586    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.367747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367621    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\" (UniqueName: \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.367747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367670    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.367747 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367731    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\" (UniqueName: \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367773    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367805    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367829    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\" (UniqueName: \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.367900    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368018    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\" (UniqueName: \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368057    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\") pod \"efa18a42-9299-4636-ba86-0e06c7d3686e\" (UID: \"efa18a42-9299-4636-ba86-0e06c7d3686e\") "
Nov 02 22:04:53.368093 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368089    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\") pod \"277619f9-50d0-41c8-a4cf-2630a53de69a\" (UID: \"277619f9-50d0-41c8-a4cf-2630a53de69a\") "
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368465    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368671    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368669    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368892    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368922    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.369198 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.368237    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.371263 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.371234    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.371360 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.371267    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "kube-api-access-wm8vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:53.371435 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.371373    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:53.371792 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.371769    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "kube-api-access-9hqkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:53.371792 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.371768    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs" pod "277619f9-50d0-41c8-a4cf-2630a53de69a" (UID: "277619f9-50d0-41c8-a4cf-2630a53de69a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:04:53.372705 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.372676    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs" pod "efa18a42-9299-4636-ba86-0e06c7d3686e" (UID: "efa18a42-9299-4636-ba86-0e06c7d3686e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:04:53.469347 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469303    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469347 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469341    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469347 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469354    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm8vc\" (UniqueName: \"kubernetes.io/projected/277619f9-50d0-41c8-a4cf-2630a53de69a-kube-api-access-wm8vc\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469364    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469374    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa18a42-9299-4636-ba86-0e06c7d3686e-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469382    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469390    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469399    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/277619f9-50d0-41c8-a4cf-2630a53de69a-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469407    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hqkn\" (UniqueName: \"kubernetes.io/projected/efa18a42-9299-4636-ba86-0e06c7d3686e-kube-api-access-9hqkn\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469416    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa18a42-9299-4636-ba86-0e06c7d3686e-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.469643 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.469424    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/277619f9-50d0-41c8-a4cf-2630a53de69a-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:04:53.817516 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.817485    4888 generic.go:356] "Generic (PLEG): container finished" podID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerID="ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30" exitCode=0
Nov 02 22:04:53.817731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.817565    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"
Nov 02 22:04:53.817731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.817582    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" event={"ID":"277619f9-50d0-41c8-a4cf-2630a53de69a","Type":"ContainerDied","Data":"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"}
Nov 02 22:04:53.817731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.817677    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt" event={"ID":"277619f9-50d0-41c8-a4cf-2630a53de69a","Type":"ContainerDied","Data":"27109b72f648a6c35e2544be53c900ee4923df1abf3a9f4b2440e64b55dd7999"}
Nov 02 22:04:53.817731 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.817717    4888 scope.go:122] "RemoveContainer" containerID="ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"
Nov 02 22:04:53.819239 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.819183    4888 generic.go:356] "Generic (PLEG): container finished" podID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerID="e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02" exitCode=0
Nov 02 22:04:53.819239 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.819219    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" event={"ID":"efa18a42-9299-4636-ba86-0e06c7d3686e","Type":"ContainerDied","Data":"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"}
Nov 02 22:04:53.819239 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.819233    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"
Nov 02 22:04:53.819416 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.819257    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr" event={"ID":"efa18a42-9299-4636-ba86-0e06c7d3686e","Type":"ContainerDied","Data":"c015afb2ffe9cbf5c931861a7510f42635695b2e73e6861f06cd95b722eb1d41"}
Nov 02 22:04:53.832715 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.832692    4888 scope.go:122] "RemoveContainer" containerID="ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"
Nov 02 22:04:53.833307 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:53.833199    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30\": container with ID starting with ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30 not found: ID does not exist" containerID="ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"
Nov 02 22:04:53.833307 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.833245    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30"} err="failed to get container status \"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30\": rpc error: code = NotFound desc = could not find container \"ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30\": container with ID starting with ba23892fe8908dfccf38645bed8854419dd63e9dda79132fe1dfe2194dfbed30 not found: ID does not exist"
Nov 02 22:04:53.833307 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.833270    4888 scope.go:122] "RemoveContainer" containerID="e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"
Nov 02 22:04:53.842594 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.842573    4888 scope.go:122] "RemoveContainer" containerID="e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"
Nov 02 22:04:53.842965 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:04:53.842943    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02\": container with ID starting with e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02 not found: ID does not exist" containerID="e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"
Nov 02 22:04:53.843046 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.842972    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02"} err="failed to get container status \"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02\": rpc error: code = NotFound desc = could not find container \"e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02\": container with ID starting with e864bf1a3dcdc4795a6aa67e6d700a0aab173a1b4f723d8a207a92ec92f85b02 not found: ID does not exist"
Nov 02 22:04:53.973393 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.973360    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 22:04:53.979142 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:53.979114    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/premium-w4-4e2b27-kserve-77cbdb9f8b-nx9sr"]
Nov 02 22:04:54.094165 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:54.094070    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 22:04:54.097075 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:54.097037    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4e2b27/distinct-w4-4e2b27-kserve-69df578557-rvzkt"]
Nov 02 22:04:54.614875 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:54.614074    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" path="/var/lib/kubelet/pods/277619f9-50d0-41c8-a4cf-2630a53de69a/volumes"
Nov 02 22:04:54.615555 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:04:54.614837    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" path="/var/lib/kubelet/pods/efa18a42-9299-4636-ba86-0e06c7d3686e/volumes"
Nov 02 22:05:13.628013 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:13.627978    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 22:05:13.628582 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:13.628205    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy" containerID="cri-o://9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8" gracePeriod=30
Nov 02 22:05:17.045570 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:17.045533    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"]
Nov 02 22:05:17.092791 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:17.092753    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9adafe35-cc4zf"]
Nov 02 22:05:18.598899 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.598840    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" path="/var/lib/kubelet/pods/ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5/volumes"
Nov 02 22:05:18.883285 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.883258    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 22:05:18.930390 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930342    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930390 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930395    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930438    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930465    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930517    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930566    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930594    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930648 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930636    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930992 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930687    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\") pod \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\" (UID: \"f693530c-15d8-4dc1-af8f-69e54a31d6ee\") "
Nov 02 22:05:18.930992 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930703    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:18.930992 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930702    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:18.930992 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930961    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:18.930992 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.930972    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:18.931243 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.931216    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:18.931243 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.931238    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:18.931352 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.931255    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:18.931352 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.931270    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:18.931352 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.931278    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:05:18.932889 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.932830    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:18.933012 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.932899    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:05:18.933152 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.933136    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "kube-api-access-sfkk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:18.933216 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.933198    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token" pod "f693530c-15d8-4dc1-af8f-69e54a31d6ee" (UID: "f693530c-15d8-4dc1-af8f-69e54a31d6ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:18.947624 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.947596    4888 generic.go:356] "Generic (PLEG): container finished" podID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerID="9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8" exitCode=0
Nov 02 22:05:18.947734 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.947646    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"
Nov 02 22:05:18.947734 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.947671    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" event={"ID":"f693530c-15d8-4dc1-af8f-69e54a31d6ee","Type":"ContainerDied","Data":"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"}
Nov 02 22:05:18.947734 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.947726    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf" event={"ID":"f693530c-15d8-4dc1-af8f-69e54a31d6ee","Type":"ContainerDied","Data":"ee4190dd5c6efa37168116f16246a92dd3f22ceee3b724fb0ae359ae13f188d9"}
Nov 02 22:05:18.947888 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.947748    4888 scope.go:122] "RemoveContainer" containerID="9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"
Nov 02 22:05:18.956880 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.956844    4888 scope.go:122] "RemoveContainer" containerID="9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"
Nov 02 22:05:18.957126 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:05:18.957110    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8\": container with ID starting with 9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8 not found: ID does not exist" containerID="9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"
Nov 02 22:05:18.957188 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:18.957134    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8"} err="failed to get container status \"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8\": rpc error: code = NotFound desc = could not find container \"9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8\": container with ID starting with 9d59b894f4afab24348dcc9329cc5b90ff9683c21bdf1fb36ddea58efa7ed0a8 not found: ID does not exist"
Nov 02 22:05:19.032139 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.032109    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:19.032139 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.032137    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:19.032139 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.032147    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:19.032369 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.032156    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f693530c-15d8-4dc1-af8f-69e54a31d6ee-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:19.032369 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.032166    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfkk8\" (UniqueName: \"kubernetes.io/projected/f693530c-15d8-4dc1-af8f-69e54a31d6ee-kube-api-access-sfkk8\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:19.146988 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.146896    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 22:05:19.153353 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:19.153328    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf"]
Nov 02 22:05:20.598951 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:20.598910    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" path="/var/lib/kubelet/pods/f693530c-15d8-4dc1-af8f-69e54a31d6ee/volumes"
Nov 02 22:05:22.261031 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:05:22.260991    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:05:23.553448 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:23.553405    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 22:05:23.553982 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:23.553674    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy" containerID="cri-o://2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6" gracePeriod=30
Nov 02 22:05:28.825775 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.825702    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 22:05:28.930915 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.930874    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.930940    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.930989    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931057    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931074    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931093    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931115    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931295    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931338    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\") pod \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\" (UID: \"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91\") "
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931187    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931359    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:28.931460 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931334    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931468    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931571    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931696    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-data\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931720    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-credential-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931736    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istiod-ca-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:28.931750 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931750    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-socket\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:28.932118 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.931763    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-workload-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:28.933423 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.933395    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:28.933536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.933415    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:28.933536 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.933515    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "kube-api-access-8q9xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:28.933622 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:28.933560    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo" pod "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" (UID: "eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:05:29.000920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.000888    4888 generic.go:356] "Generic (PLEG): container finished" podID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerID="2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6" exitCode=0
Nov 02 22:05:29.001092 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.000986    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"
Nov 02 22:05:29.001092 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.000982    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" event={"ID":"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91","Type":"ContainerDied","Data":"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"}
Nov 02 22:05:29.001182 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.001110    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq" event={"ID":"eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91","Type":"ContainerDied","Data":"32bb2b5e194c0ed0563612ebfb0ac849ab27bd9f38cca88910df7947703f74c4"}
Nov 02 22:05:29.001182 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.001142    4888 scope.go:122] "RemoveContainer" containerID="2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"
Nov 02 22:05:29.010833 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.010811    4888 scope.go:122] "RemoveContainer" containerID="2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"
Nov 02 22:05:29.011120 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:05:29.011100    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6\": container with ID starting with 2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6 not found: ID does not exist" containerID="2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"
Nov 02 22:05:29.011179 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.011130    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6"} err="failed to get container status \"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6\": rpc error: code = NotFound desc = could not find container \"2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6\": container with ID starting with 2df556a096bba7fa6d1ef7cce5280ce85b52a2b135def611062c5425565318f6 not found: ID does not exist"
Nov 02 22:05:29.032245 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.032209    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-envoy\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:29.032245 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.032239    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-podinfo\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:29.032474 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.032252    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q9xx\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-kube-api-access-8q9xx\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:29.032474 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.032262    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91-istio-token\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:05:29.156351 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.156314    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 22:05:29.160673 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:29.160647    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq"]
Nov 02 22:05:30.599464 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:05:30.599429    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" path="/var/lib/kubelet/pods/eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91/volumes"
Nov 02 22:05:52.260243 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:05:52.260184    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:06:22.260293 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:06:22.260204    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:06:52.259966 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:06:52.259927    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:06:52.480926 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:06:52.480887    4888 scope.go:122] "RemoveContainer" containerID="ee9ff71aac7c44d33e6d3de29ee9ce75d7ca3357653f2f1ef567d395014653ce"
Nov 02 22:07:22.258754 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:07:22.258709    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:07:51.065809 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:07:51.065714    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:07:51.075547 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:07:51.075520    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:07:52.258936 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:07:52.258899    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:08:22.263200 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:08:22.263159    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:08:52.259940 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:08:52.259903    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:09:22.259764 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:09:22.259720    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:09:52.260305 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:09:52.260263    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:09:54.716543 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:09:54.716504    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 22:10:01.665638 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:01.665600    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 22:10:06.665956 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:06.665905    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing" containerID="cri-o://fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a" gracePeriod=25
Nov 02 22:10:09.736812 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:09.736771    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main" containerID="cri-o://24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5" gracePeriod=45
Nov 02 22:10:09.981333 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:09.981302    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 22:10:10.059167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059084    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\" (UniqueName: \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059129    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059413 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059178    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\" (UniqueName: \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059413 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059204    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059413 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059244    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059413 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059329    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\") pod \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\" (UID: \"c96e38d2-104d-4fe7-bc6d-6407dc38acca\") "
Nov 02 22:10:10.059642 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059481    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:10.059642 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059573    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:10.059740 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059659    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:10.059740 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059724    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-home\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.059830 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.059744    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-model-cache\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.061456 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.061430    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:10.061559 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.061457    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "kube-api-access-zx6mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:10.061559 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.061481    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs" pod "c96e38d2-104d-4fe7-bc6d-6407dc38acca" (UID: "c96e38d2-104d-4fe7-bc6d-6407dc38acca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:10.161127 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.161088    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tls-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.161127 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.161118    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-tmp-dir\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.161127 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.161127    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96e38d2-104d-4fe7-bc6d-6407dc38acca-dshm\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.161127 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.161137    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx6mr\" (UniqueName: \"kubernetes.io/projected/c96e38d2-104d-4fe7-bc6d-6407dc38acca-kube-api-access-zx6mr\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:10.333973 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.333881    4888 generic.go:356] "Generic (PLEG): container finished" podID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerID="24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5" exitCode=0
Nov 02 22:10:10.333973 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.333920    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"
Nov 02 22:10:10.333973 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.333950    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" event={"ID":"c96e38d2-104d-4fe7-bc6d-6407dc38acca","Type":"ContainerDied","Data":"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"}
Nov 02 22:10:10.334228 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.334003    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh" event={"ID":"c96e38d2-104d-4fe7-bc6d-6407dc38acca","Type":"ContainerDied","Data":"6f594a5d6f299eae01f7fe41c36d0ef7c23848d15c3b266eb9605bfe9770fda7"}
Nov 02 22:10:10.334228 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.334024    4888 scope.go:122] "RemoveContainer" containerID="24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"
Nov 02 22:10:10.344258 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.344235    4888 scope.go:122] "RemoveContainer" containerID="24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"
Nov 02 22:10:10.344620 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:10.344602    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5\": container with ID starting with 24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5 not found: ID does not exist" containerID="24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"
Nov 02 22:10:10.344692 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.344628    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5"} err="failed to get container status \"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5\": rpc error: code = NotFound desc = could not find container \"24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5\": container with ID starting with 24bb7809fa64c86899607fffdb30cae03663f453ccffce5f6d258934c014d3c5 not found: ID does not exist"
Nov 02 22:10:10.488403 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.488362    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 22:10:10.490004 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.489985    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-284b8e/simulator-w6-284b8e-kserve-7669b96c64-sd5xh"]
Nov 02 22:10:10.606449 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:10.606367    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" path="/var/lib/kubelet/pods/c96e38d2-104d-4fe7-bc6d-6407dc38acca/volumes"
Nov 02 22:10:14.867528 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:14.867491    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 22:10:15.359206 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.359156    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" podUID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" containerName="maas-api" containerID="cri-o://f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0" gracePeriod=30
Nov 02 22:10:15.620968 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.620909    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 22:10:15.710394 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.710365    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") pod \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") "
Nov 02 22:10:15.710599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.710428    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\") pod \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") "
Nov 02 22:10:15.710599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.710474    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\") pod \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") "
Nov 02 22:10:15.710599 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.710561    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\") pod \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\" (UID: \"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed\") "
Nov 02 22:10:15.710920 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.710891    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca" pod "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" (UID: "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:15.712797 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.712769    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg" pod "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" (UID: "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed"). InnerVolumeSpecName "kube-api-access-j7dpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:15.713076 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.713058    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls" pod "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" (UID: "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:15.713128 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.713099    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs" pod "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" (UID: "6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:15.811885 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.811831    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:15.811885 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.811883    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:15.812102 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.811898    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:15.812102 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.811913    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed-kube-api-access-j7dpg\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:15.824837 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:15.824808    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:10:16.364990 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.364954    4888 generic.go:356] "Generic (PLEG): container finished" podID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" containerID="f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0" exitCode=0
Nov 02 22:10:16.365396 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.365056    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"
Nov 02 22:10:16.365396 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.365067    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" event={"ID":"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed","Type":"ContainerDied","Data":"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"}
Nov 02 22:10:16.365396 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.365113    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp" event={"ID":"6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed","Type":"ContainerDied","Data":"12abff4cc4c8af8b3a4a2a6d87f9ebdd2a971b4a6984b5d03dd6b96417ed788a"}
Nov 02 22:10:16.365396 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.365137    4888 scope.go:122] "RemoveContainer" containerID="f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"
Nov 02 22:10:16.374541 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.374520    4888 scope.go:122] "RemoveContainer" containerID="f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"
Nov 02 22:10:16.374886 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:16.374863    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0\": container with ID starting with f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0 not found: ID does not exist" containerID="f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"
Nov 02 22:10:16.374963 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.374894    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0"} err="failed to get container status \"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0\": rpc error: code = NotFound desc = could not find container \"f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0\": container with ID starting with f3e2aed28e817293ba5441e77aab495df6f8af6da84c9e2377515b1948042bb0 not found: ID does not exist"
Nov 02 22:10:16.532685 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.532650    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 22:10:16.580025 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.579982    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-284b8e-66f5779944-6h7jp"]
Nov 02 22:10:16.598348 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:16.598315    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" path="/var/lib/kubelet/pods/6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed/volumes"
Nov 02 22:10:22.258815 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:22.258764    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:10:25.824662 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:25.824617    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:10:26.920890 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:26.920859    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 22:10:27.015168 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.015088    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\") pod \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") "
Nov 02 22:10:27.015168 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.015144    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\" (UniqueName: \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\") pod \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\" (UID: \"ef9df833-90ea-437b-aa0a-a23e7a181bb6\") "
Nov 02 22:10:27.015517 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.015496    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume" pod "ef9df833-90ea-437b-aa0a-a23e7a181bb6" (UID: "ef9df833-90ea-437b-aa0a-a23e7a181bb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:27.017232 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.017212    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz" pod "ef9df833-90ea-437b-aa0a-a23e7a181bb6" (UID: "ef9df833-90ea-437b-aa0a-a23e7a181bb6"). InnerVolumeSpecName "kube-api-access-fb5fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:27.116304 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.116265    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9df833-90ea-437b-aa0a-a23e7a181bb6-plugins-config-volume\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:27.116304 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.116305    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb5fz\" (UniqueName: \"kubernetes.io/projected/ef9df833-90ea-437b-aa0a-a23e7a181bb6-kube-api-access-fb5fz\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:27.419864 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.419817    4888 generic.go:356] "Generic (PLEG): container finished" podID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerID="fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a" exitCode=0
Nov 02 22:10:27.420037 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.419958    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"
Nov 02 22:10:27.420037 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.419989    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" event={"ID":"ef9df833-90ea-437b-aa0a-a23e7a181bb6","Type":"ContainerDied","Data":"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"}
Nov 02 22:10:27.420131 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.420041    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t" event={"ID":"ef9df833-90ea-437b-aa0a-a23e7a181bb6","Type":"ContainerDied","Data":"e263e2d7afbe5bbb127c219e35c1d60f26727cc862bf7daf1b3c1cf93291fa22"}
Nov 02 22:10:27.420131 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.420064    4888 scope.go:122] "RemoveContainer" containerID="fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"
Nov 02 22:10:27.429140 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.429126    4888 scope.go:122] "RemoveContainer" containerID="fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"
Nov 02 22:10:27.429416 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:27.429389    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a\": container with ID starting with fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a not found: ID does not exist" containerID="fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"
Nov 02 22:10:27.429506 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.429423    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a"} err="failed to get container status \"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a\": rpc error: code = NotFound desc = could not find container \"fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a\": container with ID starting with fe99bd0fbdce9ff67a157f8a07e9f74f8849f79dbd8e3f9051e33cbf442b847a not found: ID does not exist"
Nov 02 22:10:27.574104 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.574065    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 22:10:27.576124 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:27.576100    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t"]
Nov 02 22:10:28.597982 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:28.597948    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" path="/var/lib/kubelet/pods/ef9df833-90ea-437b-aa0a-a23e7a181bb6/volumes"
Nov 02 22:10:28.790482 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:28.790449    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 22:10:28.790662 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:28.790623    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager" containerID="cri-o://0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786" gracePeriod=30
Nov 02 22:10:29.042343 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.042316    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 22:10:29.133103 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.133011    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\") pod \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") "
Nov 02 22:10:29.133103 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.133070    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\" (UniqueName: \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\") pod \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") "
Nov 02 22:10:29.133304 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.133108    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\") pod \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\" (UID: \"4f7f5139-0479-4321-9bf7-49f1dd103ad2\") "
Nov 02 22:10:29.135328 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.135290    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs" pod "4f7f5139-0479-4321-9bf7-49f1dd103ad2" (UID: "4f7f5139-0479-4321-9bf7-49f1dd103ad2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:29.135465 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.135299    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert" pod "4f7f5139-0479-4321-9bf7-49f1dd103ad2" (UID: "4f7f5139-0479-4321-9bf7-49f1dd103ad2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:29.135465 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.135366    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd" pod "4f7f5139-0479-4321-9bf7-49f1dd103ad2" (UID: "4f7f5139-0479-4321-9bf7-49f1dd103ad2"). InnerVolumeSpecName "kube-api-access-9ggnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:29.234167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.234124    4888 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-webhook-cert\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:29.234167 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.234163    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ggnd\" (UniqueName: \"kubernetes.io/projected/4f7f5139-0479-4321-9bf7-49f1dd103ad2-kube-api-access-9ggnd\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:29.234380 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.234178    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f7f5139-0479-4321-9bf7-49f1dd103ad2-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:10:29.432418 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.432334    4888 generic.go:356] "Generic (PLEG): container finished" podID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerID="0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786" exitCode=0
Nov 02 22:10:29.432418 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.432381    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerDied","Data":"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"}
Nov 02 22:10:29.432418 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.432388    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl"
Nov 02 22:10:29.432693 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.432435    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f8dffcbc-cnnfl" event={"ID":"4f7f5139-0479-4321-9bf7-49f1dd103ad2","Type":"ContainerDied","Data":"0f7751b8b23d848b922416c315b3dfb24837ac62b6ea8381d63f7fa5f6574454"}
Nov 02 22:10:29.432693 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.432465    4888 scope.go:122] "RemoveContainer" containerID="0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"
Nov 02 22:10:29.441803 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.441782    4888 scope.go:122] "RemoveContainer" containerID="f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"
Nov 02 22:10:29.450189 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.450170    4888 scope.go:122] "RemoveContainer" containerID="0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"
Nov 02 22:10:29.450490 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:29.450469    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786\": container with ID starting with 0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786 not found: ID does not exist" containerID="0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"
Nov 02 22:10:29.450572 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.450505    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786"} err="failed to get container status \"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786\": rpc error: code = NotFound desc = could not find container \"0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786\": container with ID starting with 0bcc806e415581abac01f57b8a137b484ced4decb229e8ef13fd6e7ea8cb7786 not found: ID does not exist"
Nov 02 22:10:29.450572 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.450531    4888 scope.go:122] "RemoveContainer" containerID="f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"
Nov 02 22:10:29.450811 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:29.450782    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852\": container with ID starting with f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852 not found: ID does not exist" containerID="f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"
Nov 02 22:10:29.450875 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.450819    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852"} err="failed to get container status \"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852\": rpc error: code = NotFound desc = could not find container \"f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852\": container with ID starting with f68490a28b49e1e924d9b5aeb94690fd2c9cb48f1aa68925593bdba0a9950852 not found: ID does not exist"
Nov 02 22:10:29.589036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.589000    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 22:10:29.636810 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:29.636775    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9f8dffcbc-cnnfl"]
Nov 02 22:10:30.604211 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:10:30.604178    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" path="/var/lib/kubelet/pods/4f7f5139-0479-4321-9bf7-49f1dd103ad2/volumes"
Nov 02 22:10:52.259157 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:10:52.259118    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:11:22.259812 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:11:22.259774    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:11:52.258765 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:11:52.258725    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:12:22.263574 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:12:22.263537    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:12:27.151980 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.151942    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"]
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152283    4888 cpu_manager.go:414] "containerMap: removing container" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152294    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152313    4888 cpu_manager.go:414] "containerMap: removing container" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152318    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152325    4888 cpu_manager.go:414] "containerMap: removing container" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152330    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152337    4888 cpu_manager.go:414] "containerMap: removing container" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152341    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152349    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152354    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152359    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152365    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152374    4888 cpu_manager.go:414] "containerMap: removing container" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152379    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152397    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" containerName="maas-api"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152401    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" containerName="maas-api"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152410    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2839fbe0-7269-435f-90cf-23f5eef50384" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152415    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2839fbe0-7269-435f-90cf-23f5eef50384" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152422    4888 cpu_manager.go:414] "containerMap: removing container" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152426    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152432    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152436    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152443    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152447    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152453    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152458    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152466    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152470    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152476    4888 cpu_manager.go:414] "containerMap: removing container" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152481    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152487    4888 cpu_manager.go:414] "containerMap: removing container" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152491    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152500    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" containerName="revoke-keys"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152505    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" containerName="revoke-keys"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152511    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152515    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152521    4888 cpu_manager.go:414] "containerMap: removing container" podUID="36ce2bd1-c012-44f1-85f9-9311fce8249c" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152526    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36ce2bd1-c012-44f1-85f9-9311fce8249c" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152533    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152537    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152542    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152547    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152553    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152557    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152564    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152569    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152575    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152579    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152587    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152592    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152599    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152604    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152610    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152614    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152621    4888 cpu_manager.go:414] "containerMap: removing container" podUID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152625    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" containerName="payload-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152631    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152636    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152643    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152647    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152654    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152658    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152665    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152670    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152677    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" containerName="payload-pre-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152682    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" containerName="payload-pre-processing"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152689    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1a5476b7-97d8-48be-99de-37fd520e0f8c" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152693    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5476b7-97d8-48be-99de-37fd520e0f8c" containerName="main"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152699    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152704    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152711    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152715    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy"
Nov 02 22:12:27.152671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152722    4888 cpu_manager.go:414] "containerMap: removing container" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152726    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152732    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152736    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152743    4888 cpu_manager.go:414] "containerMap: removing container" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152747    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152753    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152758    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152763    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152768    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152776    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152780    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152787    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152791    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152796    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152799    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152806    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152811    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152816    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152820    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152825    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152829    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152888    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152895    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6974c5c3-ffb3-45f6-98c9-ff0cbb2c70ed" containerName="maas-api"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152903    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2839fbe0-7269-435f-90cf-23f5eef50384" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152909    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152914    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152919    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152925    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152929    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1921f4a1-4fcf-4a87-9600-de8bd25ce0be" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152937    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="389b1624-08e6-4dc3-bff7-397abbea06f7" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152943    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="277619f9-50d0-41c8-a4cf-2630a53de69a" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152948    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebe8f6b2-4ecd-4282-9cfb-da6b0e6b04a5" containerName="revoke-keys"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152954    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152959    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="36ce2bd1-c012-44f1-85f9-9311fce8249c" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152966    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152970    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152976    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152981    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96e38d2-104d-4fe7-bc6d-6407dc38acca" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152986    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b752e56-1a34-417a-9847-e99570238998" containerName="payload-pre-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152990    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff96a539-9533-4745-b9ac-6c0c58a0f49f" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.152995    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153000    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9990587-21c7-4653-ba57-46a3be01d996" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153005    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d894546-4031-4d92-80eb-7d5f60fb59c1" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153009    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153014    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="13e03ac2-be04-4d3e-ac44-f319bfe2e789" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153019    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153024    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153030    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea1b6e79-6644-41c6-ba68-f3d77bf356d3" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153035    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8f964ac-042b-4dfc-b166-2776d59f3673" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153039    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a2fc1de-91b7-4350-ac03-c63922a3af6c" containerName="payload-pre-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153044    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5476b7-97d8-48be-99de-37fd520e0f8c" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153049    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153053    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153059    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f693530c-15d8-4dc1-af8f-69e54a31d6ee" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153064    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa18a42-9299-4636-ba86-0e06c7d3686e" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153068    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153074    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb7c87f6-5c46-4a9c-8d8d-c38a65e94b91" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153079    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153082    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2da2cc5d-90d8-4a6f-9662-6e9b1ca4de66" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153087    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153092    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153096    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b27adf3-f506-4702-92cd-becfd90326a3" containerName="main"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153101    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3e4da4c-9597-4076-9c20-6556a1c6d009" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153107    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef9df833-90ea-437b-aa0a-a23e7a181bb6" containerName="payload-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153111    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b1cc552-3f30-40b9-a564-01fca3fe3440" containerName="payload-pre-processing"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153116    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153121    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="851f0db2-7ab7-477a-95e9-310d2516314a" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153127    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="584e1c18-3c68-4e04-85a2-0a2b9e72354d" containerName="cleanup"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153131    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:12:27.154971 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.153135    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:12:27.157386 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.156269    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.255480 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.255450    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"]
Nov 02 22:12:27.289675 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289637    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c96b157-58a5-4ac8-a767-269122f12e84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.289890 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289745    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.289890 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289816    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.289890 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289837    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.290009 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289918    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.290009 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289944    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.290009 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.289978    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86kjt\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-kube-api-access-86kjt\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.290107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.290053    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c96b157-58a5-4ac8-a767-269122f12e84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.290146 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.290108    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391129 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391089    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391129 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391133    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391388 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391165    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86kjt\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-kube-api-access-86kjt\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391388 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391184    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c96b157-58a5-4ac8-a767-269122f12e84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391388 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391204    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391555 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391379    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c96b157-58a5-4ac8-a767-269122f12e84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391555 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391432    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391555 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391479    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391555 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391506    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.391835 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391806    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.392066 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.392040    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c96b157-58a5-4ac8-a767-269122f12e84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.392175 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391926    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.392175 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.391913    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.392175 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.392116    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.393627 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.393602    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c96b157-58a5-4ac8-a767-269122f12e84-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.393896 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.393877    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c96b157-58a5-4ac8-a767-269122f12e84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.445771 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.445739    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.445955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.445785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86kjt\" (UniqueName: \"kubernetes.io/projected/0c96b157-58a5-4ac8-a767-269122f12e84-kube-api-access-86kjt\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6wt7k\" (UID: \"0c96b157-58a5-4ac8-a767-269122f12e84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.468661 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.468629    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:27.881468 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.881440    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"]
Nov 02 22:12:27.882619 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:12:27.882590    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c96b157_58a5_4ac8_a767_269122f12e84.slice/crio-55f46a1a237fdaf640cefe8a75262fef2a0b94ed8d512db784c84cb86d305caf WatchSource:0}: Error finding container 55f46a1a237fdaf640cefe8a75262fef2a0b94ed8d512db784c84cb86d305caf: Status 404 returned error can't find the container with id 55f46a1a237fdaf640cefe8a75262fef2a0b94ed8d512db784c84cb86d305caf
Nov 02 22:12:27.884803 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.884769    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:27.884922 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.884869    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:27.884989 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.884917    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:27.959325 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.959281    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" event={"ID":"0c96b157-58a5-4ac8-a767-269122f12e84","Type":"ContainerStarted","Data":"f3f1a06e9538e7b65a1b3bae5b0b9f2e1c84c84f81d49611588e68cc8c9130f6"}
Nov 02 22:12:27.959521 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:27.959334    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" event={"ID":"0c96b157-58a5-4ac8-a767-269122f12e84","Type":"ContainerStarted","Data":"55f46a1a237fdaf640cefe8a75262fef2a0b94ed8d512db784c84cb86d305caf"}
Nov 02 22:12:28.069712 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:28.069656    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podStartSLOduration=1.069640882 podStartE2EDuration="1.069640882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:28.068445808 +0000 UTC m=+3577.612237725" watchObservedRunningTime="2026-11-02 22:12:28.069640882 +0000 UTC m=+3577.613432883"
Nov 02 22:12:28.468929 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:28.468884    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:28.470416 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:28.470389    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:28.470553 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:28.470444    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:29.469504 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:29.469462    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:29.469944 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:29.469521    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:30.469550 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:30.469515    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:30.469994 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:30.469567    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:31.469725 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:31.469689    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:31.470171 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:31.469755    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:32.469446 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:32.469408    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:32.469641 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:32.469464    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:33.469424 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:33.469385    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:33.469837 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:33.469443    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:34.469207 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:34.469171    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:12:34.469379 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:34.469235    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 02 22:12:35.990124 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:35.990084    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:35.990742 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:35.990688    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:35.991586 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:35.991563    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k"
Nov 02 22:12:51.103259 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:51.103230    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:12:51.105831 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:12:51.105807    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:12:52.258839 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:12:52.258803    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:13:12.487294 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487256    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"]
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487631    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487642    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b23d49a-ff76-4d09-9b0d-ea60fbf037cf" containerName="istio-proxy"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487656    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487661    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f788d02c-acc8-49e5-ab6e-3087c6f0f6d3" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487668    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487673    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487684    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487689    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487706    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487711    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ca68326-bb48-4e77-bc54-8d05a2913ba8" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487723    4888 cpu_manager.go:414] "containerMap: removing container" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487728    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="accd6e5a-552b-4e7a-a121-5ae12c683075" containerName="istio-proxy"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487787    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad514094-ac08-456c-9b01-0c4f4d6242cd" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487797    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f7f5139-0479-4321-9bf7-49f1dd103ad2" containerName="manager"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487806    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b557a0b0-277a-4e57-9646-ccf99619b700" containerName="cleanup"
Nov 02 22:13:12.487811 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.487812    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd2f7635-8670-46f4-a369-6bd8f552ddd1" containerName="cleanup"
Nov 02 22:13:12.491160 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.491138    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.591110 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.591074    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"]
Nov 02 22:13:12.632498 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632462    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632498 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632502    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632720 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632541    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/183661a0-8313-4cc5-a432-6a20e1778f46-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632720 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632597    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/183661a0-8313-4cc5-a432-6a20e1778f46-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632720 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632670    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632829 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632748    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632829 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632770    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9x94\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-kube-api-access-p9x94\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632829 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632794    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.632972 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.632835    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734268 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734216    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/183661a0-8313-4cc5-a432-6a20e1778f46-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734268 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734271    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734306    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734323    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9x94\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-kube-api-access-p9x94\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734346    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734431    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734499    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734576 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734524    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734952 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734596    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/183661a0-8313-4cc5-a432-6a20e1778f46-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.734952 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.734874    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.735076 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.735054    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-workload-certs\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.735138 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.735122    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-credential-socket\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.735313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.735293    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-data\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.735400 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.735378    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/183661a0-8313-4cc5-a432-6a20e1778f46-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.736798 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.736778    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/183661a0-8313-4cc5-a432-6a20e1778f46-istio-envoy\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.737172 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.737149    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/183661a0-8313-4cc5-a432-6a20e1778f46-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.788914 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.788822    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-istio-token\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.789049 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.788958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9x94\" (UniqueName: \"kubernetes.io/projected/183661a0-8313-4cc5-a432-6a20e1778f46-kube-api-access-p9x94\") pod \"maas-default-gateway-openshift-default-77bfb58f8c-6l5rt\" (UID: \"183661a0-8313-4cc5-a432-6a20e1778f46\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:12.803800 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:12.803765    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:13.216173 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:13.216140    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"]
Nov 02 22:13:13.217546 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:13:13.217517    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183661a0_8313_4cc5_a432_6a20e1778f46.slice/crio-d5cb30dbc4c08caf3676971bb593a7979c6445637957a46b68644ec067ef25d8 WatchSource:0}: Error finding container d5cb30dbc4c08caf3676971bb593a7979c6445637957a46b68644ec067ef25d8: Status 404 returned error can't find the container with id d5cb30dbc4c08caf3676971bb593a7979c6445637957a46b68644ec067ef25d8
Nov 02 22:13:13.219792 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:13.219760    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:13:13.219928 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:13.219827    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:13:13.219928 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:13.219888    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:13:14.174437 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.174396    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" event={"ID":"183661a0-8313-4cc5-a432-6a20e1778f46","Type":"ContainerStarted","Data":"29ba441a0220d8507764221cf01bbeff7553d2e28c66a16e01d19a9b444509f0"}
Nov 02 22:13:14.174878 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.174483    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" event={"ID":"183661a0-8313-4cc5-a432-6a20e1778f46","Type":"ContainerStarted","Data":"d5cb30dbc4c08caf3676971bb593a7979c6445637957a46b68644ec067ef25d8"}
Nov 02 22:13:14.313871 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.310189    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podStartSLOduration=2.310171114 podStartE2EDuration="2.310171114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:14.309170416 +0000 UTC m=+3623.852962331" watchObservedRunningTime="2026-11-02 22:13:14.310171114 +0000 UTC m=+3623.853963031"
Nov 02 22:13:14.804504 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.804459    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:14.806246 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.806215    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:14.806383 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:14.806283    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:15.804373 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:15.804335    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:15.804780 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:15.804403    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:16.804741 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:16.804704    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:16.805210 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:16.804762    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:17.804426 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:17.804390    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:17.804616 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:17.804454    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:18.804618 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:18.804577    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:18.805145 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:18.804635    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:19.804606 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:19.804560    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:19.804791 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:19.804634    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:20.804618 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:20.804578    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused" start-of-body=
Nov 02 22:13:20.804806 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:20.804639    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": dial tcp 10.132.0.153:15021: connect: connection refused"
Nov 02 22:13:21.808499 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:21.808466    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:21.808922 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:21.808774    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:21.809727 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:13:21.809704    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt"
Nov 02 22:13:22.259895 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:13:22.259830    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:13:52.259277 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:13:52.259231    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:14:22.033872 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:14:22.032967    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:14:22.033872 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:14:22.033062    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k" podUID="0c96b157-58a5-4ac8-a767-269122f12e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:14:22.281687 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:14:22.281637    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:14:52.259666 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:14:52.259624    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:15:22.259506 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:15:22.259466    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:15:39.097490 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:15:39.097440    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:15:39.098036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:15:39.097520    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw" podUID="edb83808-c965-4a5c-bf76-10770370a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:15:52.260136 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:15:52.260075    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:16:22.305311 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:16:22.305274    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:16:22.875036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:16:22.874980    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:16:22.875380 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:16:22.875328    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:16:52.259112 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:16:52.259030    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:17:22.258491 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:17:22.258453    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:17:51.133063 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:17:51.133029    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:17:51.136570 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:17:51.136541    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:17:52.259720 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:17:52.259683    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:18:22.259987 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:18:22.259943    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:18:52.259530 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:18:52.259487    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:19:22.258357 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:19:22.258314    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:19:52.258433 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:19:52.258326    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:20:22.258954 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:20:22.258913    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:20:52.259157 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:20:52.259112    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:21:22.259733 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:21:22.259692    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:21:52.258998 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:21:52.258952    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:22:22.258999 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:22:22.258956    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:22:51.165395 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:22:51.165315    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:22:51.168969 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:22:51.168944    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:22:52.259537 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:22:52.259500    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:23:22.266072 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:23:22.266022    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:23:52.260118 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:23:52.260073    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:24:22.260124 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:24:22.260086    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:24:52.297966 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:24:52.297927    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:24:52.867873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:24:52.866988    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:52.867873 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:24:52.867102    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:25:07.856062 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:25:07.856000    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:25:07.856800 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:25:07.856755    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt" podUID="183661a0-8313-4cc5-a432-6a20e1778f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.153:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:25:22.260497 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:25:22.260454    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:25:52.259934 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:25:52.259831    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:25:59.905170 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:25:59.905126    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8455db45c4-kb682"]
Nov 02 22:25:59.908732 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:25:59.908703    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:25:59.962058 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:25:59.962019    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8455db45c4-kb682"]
Nov 02 22:26:00.066799 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.066741    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-metrics-certs\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.067044 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.066818    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4dxm\" (UniqueName: \"kubernetes.io/projected/0ae45f9f-8935-4394-8c6c-caf13317b8d0-kube-api-access-p4dxm\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.067044 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.066940    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-maas-api-tls\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.067044 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.067012    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae45f9f-8935-4394-8c6c-caf13317b8d0-openshift-service-ca\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.167694 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.167588    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dxm\" (UniqueName: \"kubernetes.io/projected/0ae45f9f-8935-4394-8c6c-caf13317b8d0-kube-api-access-p4dxm\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.167694 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.167661    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-maas-api-tls\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.167694 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.167707    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae45f9f-8935-4394-8c6c-caf13317b8d0-openshift-service-ca\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.168031 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.167742    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-metrics-certs\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.168430 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.168405    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae45f9f-8935-4394-8c6c-caf13317b8d0-openshift-service-ca\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.170195 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.170164    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-metrics-certs\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.170307 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.170220    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae45f9f-8935-4394-8c6c-caf13317b8d0-maas-api-tls\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.221385 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.221350    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4dxm\" (UniqueName: \"kubernetes.io/projected/0ae45f9f-8935-4394-8c6c-caf13317b8d0-kube-api-access-p4dxm\") pod \"maas-api-8455db45c4-kb682\" (UID: \"0ae45f9f-8935-4394-8c6c-caf13317b8d0\") " pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.521409 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.521310    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:00.890282 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.890254    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8455db45c4-kb682"]
Nov 02 22:26:00.891765 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:26:00.891735    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae45f9f_8935_4394_8c6c_caf13317b8d0.slice/crio-dcca6d70cd67f1a382662fc968a8928b088ba892f001054a6d092c36488c6246 WatchSource:0}: Error finding container dcca6d70cd67f1a382662fc968a8928b088ba892f001054a6d092c36488c6246: Status 404 returned error can't find the container with id dcca6d70cd67f1a382662fc968a8928b088ba892f001054a6d092c36488c6246
Nov 02 22:26:00.893144 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:00.893122    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:26:01.032784 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:01.032750    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682" event={"ID":"0ae45f9f-8935-4394-8c6c-caf13317b8d0","Type":"ContainerStarted","Data":"dcca6d70cd67f1a382662fc968a8928b088ba892f001054a6d092c36488c6246"}
Nov 02 22:26:02.038831 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:02.038732    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682" event={"ID":"0ae45f9f-8935-4394-8c6c-caf13317b8d0","Type":"ContainerStarted","Data":"422f0ab9a7a939bddb9454d6f83e9782b88d87c2c5cd0858e878612025824e30"}
Nov 02 22:26:02.039238 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:02.038886    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:02.148748 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:02.148693    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682" podStartSLOduration=2.280179384 podStartE2EDuration="3.148678463s" totalImagesPullingTime="868.499079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:26:02.147418734 +0000 UTC m=+4391.691210641" watchObservedRunningTime="2026-11-02 22:26:02.148678463 +0000 UTC m=+4391.692470448"
Nov 02 22:26:08.048882 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:08.048837    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8455db45c4-kb682"
Nov 02 22:26:08.207791 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:08.207754    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 22:26:08.208003 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:08.207980    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" podUID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" containerName="maas-api" containerID="cri-o://a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c" gracePeriod=30
Nov 02 22:26:09.469667 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.469638    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 22:26:09.564828 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.564733    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\") pod \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") "
Nov 02 22:26:09.564828 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.564803    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\") pod \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") "
Nov 02 22:26:09.565056 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.564866    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\" (UniqueName: \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\") pod \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") "
Nov 02 22:26:09.565056 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.564907    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\") pod \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\" (UID: \"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757\") "
Nov 02 22:26:09.565248 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.565222    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca" pod "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" (UID: "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:26:09.567116 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.567092    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls" pod "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" (UID: "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:26:09.567222 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.567109    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs" pod "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" (UID: "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:26:09.567222 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.567208    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7" pod "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" (UID: "d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757"). InnerVolumeSpecName "kube-api-access-dgxw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:26:09.665485 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.665439    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-metrics-certs\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:26:09.665485 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.665477    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-openshift-service-ca\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:26:09.665763 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.665563    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-maas-api-tls\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:26:09.665763 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:09.665596    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgxw7\" (UniqueName: \"kubernetes.io/projected/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757-kube-api-access-dgxw7\") on node \"ip-10-0-130-83.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:26:10.076797 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.076759    4888 generic.go:356] "Generic (PLEG): container finished" podID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" containerID="a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c" exitCode=0
Nov 02 22:26:10.077036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.076811    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"
Nov 02 22:26:10.077036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.076830    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" event={"ID":"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757","Type":"ContainerDied","Data":"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"}
Nov 02 22:26:10.077036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.076902    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4" event={"ID":"d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757","Type":"ContainerDied","Data":"ec85b056a5bfee6582d108500921a1dbebfb6df5f086d0f97618b4c1e976dceb"}
Nov 02 22:26:10.077036 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.076924    4888 scope.go:122] "RemoveContainer" containerID="a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"
Nov 02 22:26:10.086318 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.086298    4888 scope.go:122] "RemoveContainer" containerID="a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"
Nov 02 22:26:10.086655 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:26:10.086635    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c\": container with ID starting with a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c not found: ID does not exist" containerID="a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"
Nov 02 22:26:10.086706 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.086666    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c"} err="failed to get container status \"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c\": rpc error: code = NotFound desc = could not find container \"a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c\": container with ID starting with a090e45b48dca82b54f71d37b8bd03c5172dc254d624369056dd66fe0ecb8b9c not found: ID does not exist"
Nov 02 22:26:10.231296 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.231256    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 22:26:10.278683 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.278641    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d98fc7974-fqwl4"]
Nov 02 22:26:10.598864 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:26:10.598819    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" path="/var/lib/kubelet/pods/d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757/volumes"
Nov 02 22:26:22.260576 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:26:22.260528    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:26:52.260360 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:26:52.260320    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:27:22.259870 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:27:22.259815    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:27:51.193292 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:27:51.193259    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:27:51.198680 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:27:51.198649    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:27:52.260493 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:27:52.260447    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:28:22.259535 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:28:22.259493    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:28:52.260343 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:28:52.260253    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:29:22.260266 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:29:22.260218    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:29:52.260377 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:29:52.260338    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:30:22.260040 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:30:22.259997    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:30:41.075451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:41.075413    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-w2wvv_68674783-8db7-4e17-8850-6da80b9259d8/manager/0.log"
Nov 02 22:30:41.344953 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:41.344868    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-q6cgf_c744b7f7-7c53-453f-b8cf-d99b04c61077/manager/0.log"
Nov 02 22:30:41.660196 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:41.660162    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-7r2zx_96b5de50-e2ab-438e-af63-984d3cb0e653/manager/0.log"
Nov 02 22:30:43.892371 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:43.892330    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8455db45c4-kb682_0ae45f9f-8935-4394-8c6c-caf13317b8d0/maas-api/0.log"
Nov 02 22:30:49.208651 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:49.208617    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-brtmx_d6946c73-4dad-4e5b-987b-01def81ac372/discovery/0.log"
Nov 02 22:30:49.553735 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:49.553656    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795b66c7fb-8b5vf_b342fdb0-cb2a-4481-8d94-f2e6131ed2e9/kube-auth-proxy/0.log"
Nov 02 22:30:50.201878 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:50.201836    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-6l5rt_183661a0-8313-4cc5-a432-6a20e1778f46/istio-proxy/0.log"
Nov 02 22:30:50.470043 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:50.469966    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-6wt7k_0c96b157-58a5-4ac8-a767-269122f12e84/istio-proxy/0.log"
Nov 02 22:30:51.560171 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:51.560107    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-x8dsw_edb83808-c965-4a5c-bf76-10770370a957/istio-proxy/0.log"
Nov 02 22:30:52.259733 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:30:52.259689    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:30:52.984569 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:30:52.984537    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79fb5bb755-mzvm9_a5cc6793-336a-4a83-b367-65858682f31b/router/0.log"
Nov 02 22:31:05.602106 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:05.602069    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-w2wvv_68674783-8db7-4e17-8850-6da80b9259d8/manager/0.log"
Nov 02 22:31:05.878000 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:05.877906    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-q6cgf_c744b7f7-7c53-453f-b8cf-d99b04c61077/manager/0.log"
Nov 02 22:31:06.188411 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:06.188374    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-7r2zx_96b5de50-e2ab-438e-af63-984d3cb0e653/manager/0.log"
Nov 02 22:31:07.921955 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:07.921929    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8455db45c4-kb682_0ae45f9f-8935-4394-8c6c-caf13317b8d0/maas-api/0.log"
Nov 02 22:31:12.656516 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:12.656482    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-brtmx_d6946c73-4dad-4e5b-987b-01def81ac372/discovery/0.log"
Nov 02 22:31:12.994565 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:12.994465    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795b66c7fb-8b5vf_b342fdb0-cb2a-4481-8d94-f2e6131ed2e9/kube-auth-proxy/0.log"
Nov 02 22:31:13.661004 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:13.660961    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-6l5rt_183661a0-8313-4cc5-a432-6a20e1778f46/istio-proxy/0.log"
Nov 02 22:31:13.921649 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:13.921556    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-6wt7k_0c96b157-58a5-4ac8-a767-269122f12e84/istio-proxy/0.log"
Nov 02 22:31:15.060644 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:15.060609    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77bfb58f8c-x8dsw_edb83808-c965-4a5c-bf76-10770370a957/istio-proxy/0.log"
Nov 02 22:31:16.427330 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:31:16.427286    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79fb5bb755-mzvm9_a5cc6793-336a-4a83-b367-65858682f31b/router/0.log"
Nov 02 22:31:22.259934 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:31:22.259888    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:31:52.259122 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:31:52.259033    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:32:22.259075 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:32:22.259037    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:32:33.067675 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:33.067642    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pb8vt_baec7585-b77e-43a2-924b-3ebeb54f7406/global-pull-secret-syncer/0.log"
Nov 02 22:32:33.624804 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:33.624773    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-jtg5l_d3061070-d6da-464b-b0df-56c03513f0da/connection-checker/0.log"
Nov 02 22:32:34.023825 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:34.023794    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-m9fk8_44199617-59fc-462a-9d0a-bae7f2b73599/connection-checker/0.log"
Nov 02 22:32:34.449579 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:34.449551    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-w69f9_e9c1d3c5-0f5b-4ae5-ab02-6aa9db4d1f83/connection-checker/0.log"
Nov 02 22:32:35.324387 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:35.324352    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jp6br_000770db-14e0-4f8e-aa43-2580970caff3/konnectivity-agent/0.log"
Nov 02 22:32:36.155530 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:36.155496    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-83.us-west-2.compute.internal_7a88fcfc7e7a1dfee7d96883c60ca9fa/haproxy/0.log"
Nov 02 22:32:42.427520 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.427480    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"]
Nov 02 22:32:42.427983 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.427922    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" containerName="maas-api"
Nov 02 22:32:42.427983 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.427939    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" containerName="maas-api"
Nov 02 22:32:42.428061 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.428034    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7a1e0d0-7373-4eea-8e6a-cb5f4ece5757" containerName="maas-api"
Nov 02 22:32:42.431254 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.431230    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.484237 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.484200    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nvhb8\"/\"kube-root-ca.crt\""
Nov 02 22:32:42.512903 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.512842    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjvs\" (UniqueName: \"kubernetes.io/projected/f0a92103-cf85-46d5-829a-4e997058aa7f-kube-api-access-nkjvs\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.513107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.513022    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-proc\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.513107 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.513070    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-podres\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.513234 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.513144    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-lib-modules\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.513234 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.513168    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-sys\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.531715 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.531678    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nvhb8\"/\"default-dockercfg-jq77d\""
Nov 02 22:32:42.531715 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.531678    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nvhb8\"/\"openshift-service-ca.crt\""
Nov 02 22:32:42.533617 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.533589    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"]
Nov 02 22:32:42.613615 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613574    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjvs\" (UniqueName: \"kubernetes.io/projected/f0a92103-cf85-46d5-829a-4e997058aa7f-kube-api-access-nkjvs\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613640    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-proc\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613659    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-podres\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613679    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-lib-modules\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613697    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-sys\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613781    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-podres\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613798    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-proc\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613794    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-sys\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.613876 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.613810    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0a92103-cf85-46d5-829a-4e997058aa7f-lib-modules\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.667360 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.667325    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjvs\" (UniqueName: \"kubernetes.io/projected/f0a92103-cf85-46d5-829a-4e997058aa7f-kube-api-access-nkjvs\") pod \"perf-node-gather-daemonset-xp2l5\" (UID: \"f0a92103-cf85-46d5-829a-4e997058aa7f\") " pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:42.744174 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:42.744070    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:43.108015 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:43.107892    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"]
Nov 02 22:32:43.110733 ip-10-0-130-83 kubenswrapper[4888]: W1102 22:32:43.110692    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0a92103_cf85_46d5_829a_4e997058aa7f.slice/crio-0ce67adcb28c9ae3d9274452400d807066795f3e9c83a6bccc6906ebf4f9fc91 WatchSource:0}: Error finding container 0ce67adcb28c9ae3d9274452400d807066795f3e9c83a6bccc6906ebf4f9fc91: Status 404 returned error can't find the container with id 0ce67adcb28c9ae3d9274452400d807066795f3e9c83a6bccc6906ebf4f9fc91
Nov 02 22:32:43.929209 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:43.929171    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5" event={"ID":"f0a92103-cf85-46d5-829a-4e997058aa7f","Type":"ContainerStarted","Data":"8fcb4b3c70e96f5fa703c51f2b48fb77000589ce845cc92f93873ce023533d79"}
Nov 02 22:32:43.929671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:43.929222    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5" event={"ID":"f0a92103-cf85-46d5-829a-4e997058aa7f","Type":"ContainerStarted","Data":"0ce67adcb28c9ae3d9274452400d807066795f3e9c83a6bccc6906ebf4f9fc91"}
Nov 02 22:32:43.929671 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:43.929283    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:44.035781 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:44.035726    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5" podStartSLOduration=2.035708028 podStartE2EDuration="2.035708028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:32:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:32:44.035273628 +0000 UTC m=+4793.579065545" watchObservedRunningTime="2026-11-02 22:32:44.035708028 +0000 UTC m=+4793.579499945"
Nov 02 22:32:49.944212 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:49.944175    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nvhb8/perf-node-gather-daemonset-xp2l5"
Nov 02 22:32:50.080451 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:50.080406    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-p8cfn_7b2d5efa-381e-4af3-ab22-06755fff7cf4/cluster-monitoring-operator/0.log"
Nov 02 22:32:51.222815 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:51.222783    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:32:51.230116 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:51.230088    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-79wcm_516a9d1c-5f96-4afb-9b1c-d9cf56e56703/console-operator/3.log"
Nov 02 22:32:52.260118 ip-10-0-130-83 kubenswrapper[4888]: E1102 22:32:52.260073    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sbzp5" containerName="extractor"
Nov 02 22:32:53.949309 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:53.949241    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxv6r_241d2e2a-901b-4e8f-8c85-ad365914e909/node-exporter/0.log"
Nov 02 22:32:54.374393 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:54.374315    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxv6r_241d2e2a-901b-4e8f-8c85-ad365914e909/kube-rbac-proxy/0.log"
Nov 02 22:32:54.784313 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:32:54.784285    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxv6r_241d2e2a-901b-4e8f-8c85-ad365914e909/init-textfile/0.log"
Nov 02 22:33:01.976586 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:33:01.976463    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68946565d6-kpkp7_c4d67dc9-550f-48a3-bfb6-7008fe02c7e9/telemeter-client/0.log"
Nov 02 22:33:02.393991 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:33:02.393894    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68946565d6-kpkp7_c4d67dc9-550f-48a3-bfb6-7008fe02c7e9/reload/0.log"
Nov 02 22:33:02.806072 ip-10-0-130-83 kubenswrapper[4888]: I1102 22:33:02.806009    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68946565d6-kpkp7_c4d67dc9-550f-48a3-bfb6-7008fe02c7e9/kube-rbac-proxy/0.log"